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(54) A remote control apparatus and a system therefor 



(57) A rennote conlrotler for exchanging information 
using infrared rays requests the direct transmission of 
a user interface (Ul) to a device to be controlled, such 
as a printer or a facsimile machine, or the indirect trans- 
mission through another device connected to the device 



to be controlled. When the Ul is returned upon the re- 
ceipt of the request, the menu of functions that the de- 
vice that is to be controlled can execute using the Ul is 
displayed on the screen of the renrxDte controller, and a 
command to execute a function that is selected from the 
menu is transmitted to the device that is to be controller 
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Description 

BACKGROUND OF THE INVENTION 

Field of the Invention 

The present invention relates to an information 
processing apparatus that can operate various devices 
indirectly by directly operating a remote controller, or an- 
other device connected with the remote controller, and 
a system therefor. 

Related Background Art 

A remote controller for exercising operational con- 
trol of an apparatus Irom a remote location is employed 
primarily for home electric appliances, such as video re- 
corders and air conditioners. 

In addition, remote controllers that are used for tel- 
evisions and video recorders, and remote controllers 
that are compatible with a plurality of television models 
have appeared. 

In order to operate a specific model by using a con- 
ventional remote controller that is compatible with a plu- 
rality of models, an operator must change the setup in 
the remote controller. 

Thus, as the number of compatible models has in- 
creased, setup operations have become so complicated 
that operators must accurately identify models that are 
to be controlled and must then refer to operation man- 
uals to set up remote controllers, so that the likelihood 
is great that operators will mistake when they perfonm 
setups. And when the models that are to be controlled 
are changed frequently, the setup must be repeated of- 
ten and usability will be deteriorated. 

The devices that are to be controlled by such re- 
mote controllers are limited to those that are located that 
they are inside the effective range of the control signals 
emitted by the remote controllers. 

SUMMARY OF THE INVENTION 

To resolve the above shortcomings, it is one objec- 
tive of the present invention to provide an information 
processing apparatus that can easily control a variety of 
devices, and a system therefor 

it is another objective of the present invention to 
-provide a remote control apparatus that can easily cope 
with a plurality of devices without a series of complicated 
setups being required of a user and that can control the 
devices from a remote location, and a system therefor 

It is an additional objective of the present invention 
to provide a remote control system that can control, via 
a nearby device, a device in a remote location to which 
instructions can not be directly transmitted. 

It is a further objective of the present invention to 
provide an information processing apparatus that can, 
as needed, determine an appropriate process to be per- 



formed, such as communication with another device, 
and can then perform the process. 

It is still another objective of the present invention 
to provide an information processing apparatus that can 
s acquire, from another device, information that it is to 
process, and can then process the informatbn. 

It is a still additional objective of the present inven- 
tion to provide an information processing apparatus that 
can transmit information, such as an instruction that is 
10 the result obtained from a process it performed, to an- 
other device for storage at that device. 

It is a still further objective of the present invention 
to provide an information processing apparatus that can 
control another device by employing an operation unit 
fs that controls the information processing apparatus. 

The other objective of the present invention Is to 
provide an information processing apparatus that can 
flexibly cope with changes in the operational proce- 
dures. 

20 According to one aspect, the present invention 
which achieves these objectives relates to a remote 
control apparatus comprising: 

reception means for receiving information concem- 
25 ing a device to be controlled; 

validation means for validating operating proce- 
dures for the device to be controlled; and 
transmission control means for transmitting an op- 
erating command to the device based on the oper- 
30 ating procedures that are validated. 

According to another aspect, the present invention 
which achieves these objectives relates to a remote sys- 
tem comprising: 

35 

identification means for identifying a device to be 
controlled by a remote control apparatus; and 
transmission control means for transmitting to the 
remote control apparatus information concerning a 
40 device that is identified, 

wherein the remote control apparatus transmits an 
operating command to the device in accordance 
with operating procedures based on the information 
that is received. 

45 

According to yet another aspect, the present inven- 
tion which achieves these objectives relates to a remote 
control apparatus comprising: 

50 input means for receiving an instruction; 

processing means for performing processing in ac- 
cordance with the instruction input by the input 

means; 

communication means for communicating with an- 
55 other device; and 

communication control means for permitting the 
communication means to transmit an operation in- 
struction, which has been input for another device 
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by the input means, to the device to be controlled 
according to the instruction. 

According to still another aspect, the present inven- 
tion which achieves these objectives relates to an infor- 
mation processing apparatus comprising: 

determination means for specifying a process to be 
performed; and 

processing means for performing the process spec- 
ified by the determination means. 

According to one more aspect, the present inven- 
tion which achieves these objectives relates to an infor- 
mation processing apparatus comprising: 

communication means for communicating with an 
external device; 

processing means for processing information from 
the external device that is received through the 
communication means; 

acquisition means for acquiring information from 
the external device through the communication 
means; and 

control means for permitting the processing means 
to process the information acquired by the acquisi- 
tion means. 

According to yet one more aspect, the present in- 
vention which achieves these objectives relates to an 
information processing apparatus comprising: 

communication means for communicating with an 
external device; 

processing means for processing information from 
the external device that is received through the 
communication means; 

transmission control means for transmitting to the 
external device through the communication means 
the infornnation that is processed by the processing 
means; and 

control means for storing in the external device the 
information that has been transnnitted to the exter- 
nal device through the communication means. 

According to still one more aspect, the present in- 
vention which achieves these objectives relates to an 
information processing apparatus comprising: 

operating means for performing an operation; 
input means for inputting information; 
processing means for, based on the operation, 
processing the information input by the input 
means; 

output means for outputting results obtained by the 
processing means; and 

control means for controlling another device based 
on an operation for the device performed by the op- 



eration means. 

According to an additional aspect, the present in- 
vention which achieves these objectives relates to an 
5 information processing method comprising: 

a determination step of specifying a process to be 
performed by an information processing apparatus; 
and 

10 a processing step of performing the process spec- 
ified at the determination step. 

According to yet an additional aspect, the present 
invention which achieves these objectives relates to an 
15 information processing method comprising: 

a processing step of processing information that is 
received from an external device through a commu- 
nication unit, which is used to communicate with the 
20 external device; 

an acquisition step of acquiring information through 
the communication unit from the external device; 
and 

a control step of, at the processing step, processing 
25 the information that is acquired at the acquisition 
step. 

According to still another additional aspect, the 
present invention which achieves these objectives re- 
30 lates to an information processing method comprising: 

a processing step of processing information; 
a transmission step of transmitting the information 
that has been processed at the processing step to 
35 an external device through a communication unit, 
which is used to communicate with the external de- 
vice; and 

a control step of storing in the external device the 
information that has been transmitted through the 
40 communication unit to the external device. 

According to a further aspect, the present invention 
wtiich achieves these objectives relates to an informa- 
tion processing method, for an information processing 
45 apparatus that has an operating unit for performing 
processing, comprising: 

an input step of inputting information; 
a processing step of processing, based on the op- 
50 eration, the information input at the input step; 

an output step of outputting results obtained at the 
processing step; and 

a control step of controlling another device based 
on an operation performed for the device by the op- 
55 eration unit. 

According to a still further aspect, the present in- 
vention which achieves these objectives relates to a 
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computer-readable storage medium used to store an in- 
formation processing program for controlling a compu- 
ter for processing of information, the program compris- 
ing codes for enabling the computer to perform: 

a determination step of specifying a process to be 
performed by an information processing apparatus; 
and 

a processing step of performing the process spec- 
ified at the determination step. 

According to one more additional aspect, the 
present invention which achieves these objectives re- 
lates to a compuier-readable storage medium used to 
store an inlormntion processing program tor controlling 
a computer lor pfoccsstng ot information, the program 
comprising codes tor cnribsmo the computer to perform: 

a processing slop of procossrng information that is 
received Irorn nn external device through a commu- 
nication unit wt ncti 1^ used to communicate with the 
external device 

an acquisition step of acquiring mlormation through 
the communication unit from the external device; 
and 

a control step of. at the processing step, processing 
the information that is acquired at the acquisition 
step. 

According to yet one more additional aspect, the 
present invention which achieves these objectives re- 
lates to a computer-readable storage medium used to 
store an information processing program for controlling 
a computer for processing of information, the program 
comprising codes for enabling the computer to perform: 

a processing step of processing information; 
a transmission step of transmitting the information 
that has been processed at the processing step to 
an external device through a communication unit, 
which is used to communicate with the external de- 
vice; and 

a control step of storing in the external device the 
information that has been transmitted through the 
communication unit to the external device. 

According to still one more additional aspect, the 
present invention which achieves these objectives re- 
lates to a computer-readable storage medium used to 
store an information processing program for controlling 
a computer for processing of information, the program 
comprising codes for enabling the computer to perform: 

an input step of inputting information; 
a processing step of processing, based on the op- 
eration, the information input at the input step; 
an output step ot out putting results obtained at the 
processing step; and 



a control step of controlling another device based 
on an operation performed for the device by the op- 
eration unit. 

s Other objectives and advantages besides those 
discussed above will become apparent to those skilled 
in the art during the course of the description of the pre- 
ferred embodiments of the invention that follows. In the 
description, reference is made to accompanying draw- 
10 ings, which form a part thereof, and which illustrate an 
example of the invention. Such example, however, is not 
exhaustive of the various embodiments of the invention, 
and therefore reference is made to the claims that follow 
the description for determining the scope of the inven- 
ts tion. 

BRIEF DESCRIPTION OF THE DRAWINGS 

Fig. 1 is a block diagram illustrating the arrange- 
20 ment'^of an information processing system accord- 
ing to a first embodiment of the present invention; 
Fig. 2 is a diagram illustrating the functional ar- 
rangement according to the first embodiment; 
Fig. 3 is a diagram illustrating the hardware ar- 
25 rang erne nt of the individual units according to the 
first embodiment; 

Fig. 4 is a flowchart showing the processing per- 
formed for the first embodiment; 
Fig. 5 is a diagram showing a process for selecting 
30 an optimal printer from a plurality of printers and 

performing a printing job with it; 
Fig. 6 is a diagram for explaining a case where a 
controller is used; 

Fig. 7 is a diagram showing a case where a docu- 
3S ment is read by a scanner tor transmission to a spe- 
cific printer; 

Figs. 8, 9 and 10 are diagrams illustrating a case 
where a document is read by a scanner for trans- 
mission to a specific printer that the scanner can not 
40 control; 

Fig. 11 is a diagram Illustrating a case where infor- 
mation concerning the state of a printer is distribut- 
ed; 

Fig. 12 is a flowchart showing the processing per- 
45 formed an eighth embodiment; 

Fig. 1 3 is a table containing remarks for determining 
whether or not an advance notice should be trans- 
mitted in consonance with job types and conditions; 
Figs. 14A and 14B are respectively diagrams of a 
50 case where a user is informed in advance of infor- 
mation to be filed and a case where unnecessary 
information is abandoned; 

Fig. 15 is a flowchart showing the processing for a 
ninth embodiment; 
55 Fig, 16A is a diagram showing the transmission of 
information for the ninth embodiment; 
Fig. 16B is a diagram showing the transmission of 
informatbn when an apparatus corresponding to 
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the object of a job is not present; 
Fig 17 is a diagram showing an example display 
tor a window by which an optimal method is pro- 
posed to a user; 

Fig. 18 is a flowchart showing the detailed proce- 5 
dure for job execution; 

Fig. 1 9 is a diagram showing an example of classi- 
fied information to be read; 
Fig. 20 is an example job table; 

Fig 21 is a diagram showing an example printing fo 
window; 

Fig 22 is an example job table; 

Fig 23 IS a diagram showing the contents of <file 

A> 

Fig 24 IS a diagram showing an inquiry window; 

l-tg 25 IS an example job table; 

f-ig 26 ts a diagram showing example scheduling 

Fig 27 ts a diagram showing an example user que- 
'Y 

Fty 25 lb rill example job table; 
Fig 29 IS rfn example job table; 
Fig 30 IS H diagram showing an example change in 
I!^c Gl^itu^ ol a printer; 
Fig 31 iG ci table showing printer statuses; 
Fig 32 IS an example job table; 
Fig 33 IS a diagram showing a query window; 
Fig 34 IS an example job table; 
Fig 35 IS a diagram showing an example printer 
change status 

Fig 36 IS a table for displaying planning and action 
rules: 

Fig 37 is a table for displaying planning and action 
rules: 

Fig 38 is a diagram showing actions and premise 35 
conditions for an object; 

Fig. 39 IS a table for example apparatus properties; 
Fig. 40 is a table for example apparatus statuses; 
Figs. 41 A and 41 B are diagrams showing states 
where a plurality of apparatuses are connected to 
a network: 

Fig. 42 is a diagram showing a functional arrange- 
ment for performing the processing for a fourteenth 
embodiment; 

Fig. 43 is a main flowchart showing the processing ^5 
for the fourteenth embodiment; 
Fig. 44 is a flowchart showing the processing for 
conlrolling olher apparatuses; 
Fig. 45 is a flowchart showing the processing per- 
formed by one's own apparatus; 
Fig. 46 is a diagram showing an example where a 
printer other than a designated printer is employed 
for output; 

Fig. 47 is a diagram showing an example where a 
designated printer transfers a job to another printer; 55 
Fig. 48 is a diagram showing an example where a 
user's printer transfers a job to a printer at a user's 
destination; 



Fig. 49 is a flowchart showing the processing for a 
sixteenth embodiment; 

Fig. 50 is a conceptual diagram showing the 
processing for the sixteenth embodiment; 
Fig. 51 is a diagram showing transmission of a job 
for a seventeenth embodiment; 
Fig. 52 is a flowchart showing the processing per- 
formed by a print job automatic setup unit; 
Fig. 53 is a table showing printing periods and con- 
sumption of toner for a plurality of printers for print- 
ing a single character; 

Fig. 54 is a diagram showing the contents of a print 
job and parameters designated by a user; 
Fig. 55 is a flowchart showing the processing per- 
formed by a print job automatic change unit; 
Fig. 56 is a diagram showing the contents of a print 
job and parameters designated by a user; 
Fig. 57 is a diagram showing transmission of a job 
for a nineteenth embodiment; 
Fig. 58A is a diagram for explaining an example 
where parameters are set for a print job by employ- 
ing a natural language; 

Fig. 58B is a table showing the meanings of char- 
acter strings in the parameter setup, and request 
items with which to complete the meanings; 
Fig. 59 is a diagram showing an example where pa- 
rameters of a print job are set by using a document; 
Fig. 60 is a diagram showing transmission of a job 
for a twenty-first embodiment; 
Fig. 61 is a flowch'art showing the processing per- 
formed by a print job simulation unit; 
Fig. 62 is a flowchart showing the processing per- 
formed by a printer system according to a twenty- 
second embodiment; 

Figs. 63A and 63B are diagrams illustrating the ex- 
ternal appearance of the printer system according 
to the twenty-second embodiment; 
Fig. 64 is a flowchart showing the processing per- 
formed for a twenty-third embodiment; 
Fig. 65 Is a diagram showing the functional arrange- 
ment of a system according to the twenty-third em- 
bodiment; 

Fig. 66 is a diagram showing a system for perform- 
ing the overall process according to the twenty-third 
embodiment and input/output types relative to an 
external apparatus; 

Fig. 67 is a diagram showing an example where a 

schedule is extracted from electronic mail; 

Fig. 68 is a flowchart showing specific processing 

performed for the twenty-third embodiment; 

Fig. 69 is a diagram showing an example schedule 

conflict; 

Fig. 70 is a diagram showing an example where a 
user aurally queries a system; 
Fig. 71 is a flowchart showing the processing for a 
twenty-fourth embodiment; 

Fig. 72 is a diagram showing an example rearrange- 
ment of conflicting schedules; 
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Fig. 73 is a flowchart showing the processing for re- 
planning conflicting schedules; 
Fig. 74 is a flowchart showing the processing for re- 
planning responses; 

Fig. 75 is a diagram showing an overall image in- 5 
eluding the input/output of a system according to a 
twenty-sixth embodiment; 

Fig. 76 is a flowchart showing the overall processing 
performed by the system according to the twenty- 
sixth embodiment; 

Fig. 77 is a flowchart showing the processing per- 
formed by an input management unit; 
Fig. 78 is a flowchart showing the processing per- 
formed by a core unit; 

Fig. 79 is a flowchart showing the processing per- ^5 

formed by an output management unit; 

Fig. 80 is a flowchart showing data derivation 

processing; 

Fig, 81 is a diagram showing an example document 
from which data are to be extracted; 
Figs. 82A and 82B are diagrams for explaining a 
specific block reading process; 
Fig. 83 which is composed of Figs. 83A and 83B are 
flowcharts showing determination procedures in a 
process for referring to date data; ^5 
Fig. 84 is a diagram showing transmission of infor- 
mation according to a twenty-seventh embodiment; 
Fig. 85 is a diagram showing transmission of infor- 
mation according to a twenty-eighth embodiment; 
Fig. 86 is a diagram showing transmission of infor- 30 
mation according to the twenty-eighth embodiment; 
Fig. 87 is a flowchart showing the processing for the 
twenty-eighth embodiment; 

Fig. 88 is a table showing knowledge of a common 
knowledge base; ^5 
Fig. 89 is a table showing knowledge of a common 
knowledge base in a field specified on a cover page; 
Fig. 90 is a diagram showing an example where 
voice is used to refer to processing history to per- 
form a process; 

Fig. 91 is a flowchart showing the processing per- 
formed according to a thirtieth embodiment; 
Fig. 92 is a diagram illustrating an example input 
document; 

Fig. 93 is a diagram showing the contents of a con- 45 
versation between a system and a user; 
Fig. 94 is a diagram showing an output document 
that a system prepares based on the judgement of 
the conversation; 

Fig. 95 is a flowchart for an example where a re- 
quired process is automatically performed after a 
user is queried; 

Fig. 96 is a diagram showing an example wherein 

a system is operated by a controller; 

Fig. 97 is a diagram showing an example status ^5 

monitor; 

Fig. 98 is a diagram showing an example wherein 
a controller recognizes a target model and displays 



a corresponding Ul; 

Fig. 99 is a flowchart showing the processing per- 
formed by a controller for the example wherein the 
controller recognizes a target model and displays a 
corresponding Ul; 

Fig. 100 is a flowchart showing the processing per- 
formed by a target model for the example wherein 
the controller recognizes a target model and dis- 
plays a corresponding Ul; 

Fig. 101 is a diagram showing an example wherein 
a controller receives a Ul from a target model and 
displays it; 

Fig. 102 is a flowchart showing processing per- 
formed by a controller for the example wherein the 
controller receives a Ul from a target model and dis- 
plays it; 

Fig. 103 Is a flowchart showing processing per- 
formed by a target model for the example wherein 
the controller receives a Ul from the target model 
and displays it; 

Fig. 1 04 is a diagram showing an example wherein 
a controller receives a Ul from a target model via a 
wireless LAN rather than directly, and displays it; 
Fig. 1 05 is a flowchart showing the processing per- 
formed by a controller for the example wherein the 
controller receives a Ul via a wireless LAN and dis- 
plays it; 

Fig. 106 is a flowchart showing the processing per- 
formed by a target model for the example wherein 
the controller receives a Ul via a wireless LAN and 
displays it; 

Fig. 107 is a flowchart showing the processing per- 
formed by a server for the example wherein the con- 
troller receives a Ul via a wireless LAN and displays 
it; 

Fig. 108 is a diagram showing an example wherein 
via a wireless LAN a controller receives a different 
Ul for each user and displays it; 
Fig. 109 is a flowchart showing the processing per- 
formed by a controller for the example wherein via 
a wireless LAN a controller receives a different Ul 
for each user and displays it; 
Fig. 110 is a flowchart showing the processing per- 
formed by a target model for the example wherein 
via a wireless LAN a controller receives a different 
Ul for each user and displays it; 
Fig. 111 is a flowchart showing the processing per- 
formed by a server for the example wherein via a 
wireless LAN a controller receives a different Ul for 
each user and displays it; 

Fig. 112 is a table showing a Ul determined in con- 
sonance with a target model and user identification 
information; 

Fig. 1 1 3 is a diagram illustrating an example where- 
in a model in front that is to be operated performs 
a function that the model does not have; 
Fig. 114 is a diagram illustrating an example where- 
in a model in front that is to be operated performs 
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a function that the mcxJel does not have; 
Fig. 115 is a diagrann showing an example per- 
formed when comnnunication can not be performed 
directly with a model in front that is to be operated; 
Fig. 1 1 6 is a flowchart showing the processing per- s 
formed by a controller according to a thirty-ninth 
embodiment; 

Fig. 117 is a diagram showing an example wherein 
an operation/action by a user and a time are jointly 
stored as history along with important associated io 
information; 

Fig. 118 is a flowchart for the processing performed 
by a controller according to a fortieth embodiment; 
Fig. 1 1 9 is a flowchart for the processing performed 
by a target model according to the fortieth embodi- ^5 
ment; 

Fig. 1 20 is a flowchart tor the processing performed 
by a server according to the fortieth embodiment; 
Fig. 121 is a table showing example updated history 
information; 

Fig. 1 22 is a diagram illustrating an example where- 
in an operation is performed based on history; 
Fig. 123 is a diagram illustrating the example 
wherein the operation is performed based on histo- 

Fig. 124 is a diagram showing an example structure 
for a controller; 

Fig. 125 is a diagram showing another example 
structure for a controller; 

Fig. 126 is a diagram showing an additional exam- 30 
pie structure for a controller; 
Fig. 1 27 is a diagram showing a functional arrange- 
ment according to a forty-second embodiment; 
Fig. 128 is a flowchart showing the processing for 
the forty-second embodiment; 
Figs. 129A, 129B, 129C, 129D, 129E and 129F are 
example task tables; 

Fig. 1 30 is a diagram showing an example input/ 
output apparatus that can operate another appara- 
tus; 

Fig. 1 31 is a diagram showing an example input/ 
output apparatus that can operate a system consist- 
ing of a plurality of apparatuses; 
Fig. 1 32 is a diagram illustrating example input/out- 
put apparatuses that can communicate with each 
other; 

Fig. 133 is a diagram illustrating an example input/ 
output apparatus that can acquire information from 
another apparatus; 

Fig. 1 34 Is a diagram Illustrating an example input/ 
output apparatus that can store information in an- 
other apparatus; 

Fig. 135 is a diagram illustrating an example input/ 
output apparatus that can send an Instruction to an- 
other apparatus; 

Fig. 1 36 which is composed of Figs. 1 36A and 1 36B 
are flowcharts showing the processing for the task 
<COMMUNICATE>; 



Fig. 1 37 which is composed of Figs. 1 37A and 1 37B 
are flowcharts showing the processing for the task 
<PULL>; 

Fig. 1 38 which is composed of Figs. 1 38A and 1 388 
are flowcharts showing the processing for the task 
<PUSH>; 

Fig. 1 39 which is composed of Figs. 1 39A and 1 39B 
are flowcharts showing the processing for the task 
<INDtCATE>; 

Fig. 140 which is composed of Figs. 140Aand 1408 
are flowcharts showing the processing for the task 
<RESPOND>; 

Fig. 141 is a flowchart showing the processing for 
a response task relative to an information request 
type request; 

Fig. 142 is a flowchart showing the processing for 
a response task relative to an information storage 
type request; 

Fig. 143 is a flowchart showing the processing for 
a response task relative to an instructions type re- 
quest; 

Fig. 144 is a diagram illustrating an example pull 
session; 

Fig. 145 is a diagram illustrating another example 
pull session; 

Fig. 146 is a diagram illustrating an additional ex- 
ample pull session; 

Fig. 147 is a diagram illustrating an example Hear 
On Demand session; 

Fig. 148 is a diagram illustrating a further example 
pull session; 

Fig. 149 is a diagram illustrating another example 
pull task; 

Fig. 150 is a diagram illustrating an additional ex- 
ample pull task; 

Fig. 151 is a diagram illustrating a session for a fur- 
ther example pull task; 

Fig. 152 is a diagram illustrating an example push 
task; 

Fig. 153 is a diagram illustrating an example push 
session; 

Fig. 154 is a diagram illustrating an example mail 
transmission; 

Fig. 155 is a diagram illustrating example instruc- 
tions task INDICATE session; 
Fig. 156 is a diagram illustrating another example 
instructions task INDICATE session; 
Fig. 157 Is a diagram illustrating additional example 
instructions task INDICATE session; 
Fig. 1 58 is a diagram showing the flow of an oper- 
ation foltowing the operational procedures per- 
formed for another apparatus; 
Fig. 1 59 is a diagram illustrating an example where- 
in a PC transmits a printing instruction to a printer; 
Fig, 160 is a diagram for explaining an example 
wherein a PC is emptoyed as a server terminal; 
Fig. 161 is a diagram for explaining an example 
wherein a scanner and a printer are controlled by a 
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PC; 

Fig. 162 is a diagram showing a relationship be- 
tween a print job status change and a process as- 
sociated with each status; 

Fig. 1 63 which is connposed of Figs. 1 63A and 1 63B s 
are flowcharts showing printing processing; 
Fig. 164 is a diagram showing a relationship be- 
tween a print job and its processing, and a user's 
operation; and 

Fig. 165 is a diagram showing an example display 
screen tor a user for operating a print job. 

DETAILED DESCRIPTION OF THE PREFERRED 
EMBODIMENTS 

15 

The prclcrred embodiments of the present inven- 
tion wtll now be described while referring to the accom- 
panying drawings 

(Firsl EmbodtmenlJ 

Fig 1 IS a d agram illustrating a system arrange- 
ment according to a first embodiment in which a PC 
{Personal Computer) 101 , a scanner 1 02, a color printer 
103 and a monochrome printer 104 are connected to a 25 
network 

In this embodiment when an apparatus in a system 
wherein a plurality of apparatuses are connected to- 
gether IS instructed lo execute a specific job, the appa- 
ratus analyzes Ihe object of the job by examining its con- 30 
tents, and thus ascertains which job to execute. As a 
result of the analysis, the apparatus acquires necessary 
information concerning the job and thereafter executes 
it. 

If for the execution of a job there is available a more ^5 
effective method or another and better apparatus than 
an instructed method or an assigned apparatus, an in- 
struction is issued to the better apparatus or a notice is 
transmitted to a user to propose the use of the more ef- 
fective method- Further, as a consequence of the exam- 
ination of the contents of the job the instruction may be 
rejected. 

Job types are an operation input by a user, an exe- 
cution of an instruction received from another appara- 
tus, the acquisition of results by analyzing a job, or a job 
that is newly generated by an owned apparatus during 
an idling period. When, for example, a document is to 
be read by a scanner and printed by a printer, the fol- 
lowing jobs are generated. 

(1 ) Analysis of the contents of an instruction that is 
transmitted from a user to a scanner and that is re- 
lated to a document set in the scanner; 

(2) A reading and analyzation process for a docu- 
ment, and an instruction process for instructing a 
printer to print a document that has been read, both 
of which are generated by analyzing an instruction 
from a user that is received by the scanner (and the 



fact that the document has been set in the scanner). 

(3) Generation by the scanner of a printing instruc- 
tion for a printer during the instruction process. 

(4) Generation of a printing process by the printer 
as a result of an analysis of the printing instruction 
received from the scanner 

A method for generating and managing these jobs 
will later be described in detail by referring to an eleventh 
and foltowing embodiments. 

The procedures according to which a plurality of ap- 
paratuses execute the above described processes will 
be specifically described while referring to the drawings. 

Fig. 2 is a diagram illustrating the functional ar- 
rangement for performing the processing in this embod- 
iment. A job reception unit 201 receives a job from a 
user or another apparatus. The received job is regis- 
tered in a job table. A job analysis unit 203 reads and 
analyses a job registered in the job table 202. Another 
apparatus information acquisition unit 204 acquires the 
attributes and the current states of the other apparatus- 
es that are connected. A state determination unit 207 
determines the state from the current states of the other 
apparatuses. 

An optimum planning unit 206 makes an optimal 
plan for the execution of a job. A job execution determi- 
nation unit 207 determines whether a job should be ex- 
ecuted, or when the job is to be executed, and deter- 
mines whether an owned apparatus or another appara- 
tus will execute the job. When another apparatus exe- 
cutes the job, another apparatus instruction unit 208 
sends an instruction to another apparatus for the job ex- 
ecution. When the owned apparatus executes the job, 
a job execution unit 209 executes the job. An execution 
notification unit 210 notifies a user that the execution 
instruction has been issued to another apparatus. 

Fig. 3 is a diagram showing the hardware arrange- 
ment of the units that perform the processing in this em- 
bodiment. 

An I/O 301 exchanges inpuVoutput data with exter- 
nal apparatuses. A CPU 302 executes a program and 
controls the individual units. A ROM 303 is used to store 
programs, which correspond to individual flowcharts 
that will be described later and which are to be executed 
by the CPU 302, and fixed data. A RAM 304 is used to 
temporarily store variables and intermediate data, such 
as the attributes and the current states of the other ap- 
paratuses acquired by the other apparatus information 
acquisition unit 204, that are generated during the 
processing. A program from an external source may be 
loaded into the RAM 304 and stored therein. 

Fig. 4 is a flowchart showing the processing per- 
formed for the first embodiment. 

At step 8109, the contents of the job table are ini- 
tialized. At step S110, in order to determine whether a 
job has been input, a check is performed to determine 
whether or not a job has been input by a user, a job has 
been input by a device that can be detected by the sys- 
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tem, from results obtained by analyzing the job, or a new 
job has been generated by an owned apparatus during 
idling. When a job has been input, at step S11 1 . the input 
job is added to the job table. At step S11 2. the job table 
is examined to determine whether there is job. If there 
is a job, at step S113 the job is extracted. At step S1 14 
a received instruction is analyzed, and from the infor- 
mation thus obtained, the object of the job is acquired. 
A1 step S1 15, the analyzed instruction is employed to 
search for a job that is to be executed. At step S116 a 
job that is searched for at S1 15 is registered. 

At step S117 a check is performed to determine 
whether or not there is a job that has not yet been exe- 
cuted- If there is such a job, program control advances 
to step S118. At step 8118 a check is performed to de- 
termine whether or not the job can be executed. When 
it is ascertained that the job can not be executed, pro- 
gram control returns to step SI 15 and a job that must 
be executed is searched for. At step S11 9 the job is ex- 
ecuted and program control returns thereafter to step 
3117. 

If a user sends an instruction, for example, to read 
a document using a scanner and to print the document 
using a printer it is assumed that, as a result of the anal- 
ysis, the acquisition of a printed document is the object 
of a user. In this case, a document reading and analysis 
process, which is generated by analyzing information 
received by the scanner from the user, and a process 
for instructing a printer to print the document that has 
been read are searched for as jobs. These processes 
will be described in detail during the course of the elev- 
enth and the following embodiments while referring to 
specific examples. 

Fig. 5 is a diagram showing a process for selecting 
from among a plurality of printers an optimal printer for 
performing printing. 

In Fig. 5, job information is input/transmitted as a 
command from a host computer or an instruction from 
a remote controller, or is obtained by analyzing a voice 
command issued by a user using natural language, or 
from bar code or a sentence read from a cover page. 

A device that has received the job inf omnation ana- 
lyzes the job, and then selects a printer that can execute 
a process suitable for the job and outputs the data to the 
printer. Further, when there is an instruction for notifica- 
tion, the device selects an optimal medium and dis- 
patches a notice thereto. 

Therefore, upon the receipt of a request from a user, 
the structure of the printer is referred to, and an appro- 
priate printer is selected to perform the printing. At this 
time, in consonance with the situation, the processing 
can be allocated for a plurality of printers. Further, when 
there is an instruction for notification, an optimal medium 
is selected to dispatch a notification that the printing has 
been terminated along with the name of the printer that 
has performed the printing. 

A request submitted by a user concerns paper siz- 
es, printing quality, colors and time limit (e.g.. comple- 



tion of printing by 5 o'clock). The printer configuration 
covers the printer types and the number of printers in 
the system, and their performances or their current sta- 
tuses (whether the printers are normally operated, 

5 whether there is a printing queue, or whether paper or 
toner is insufficient). As for a medium and a method 
used for notification, a notification is transmitted by 
voice through a user's terminal or by telephone, or a no- 
tification message is transmitted to a pager, or a docu- 

10 ment describing the contents of a notification is trans- 
mitted by electronic mail or by facsimile. 

A situation where the individual units perform the 
processing shown in Fig. 5 will now be described while 
referring to the flowchart in Fig. 4. 

IS First, if, at step S110, an urgent instruction for out- 
putting a specific document by five o'clock is transmitted 
to a host computer, at step S1 11 a job is added to a job 
table. At step S1 1 3, the job is extracted from the job table 
and at step S1 14 the received instruction is analyzed to 

20 obtain the object of the job. 

At step S1 15. the following jobs are searched for 
following the analysis of the instruction. 

(1 ) Acquisition of specifications (paper size, printing 
2S quality, color document, etc. ) to be used for select- 
ing a suitable printer for outputting a document, and 
of other information, such as time limit. 

(2) Selection from among the connected printers of 
a suitable printer for specifications (1). 

30 (3) Examination of the selected printer to determine 

whether or not any obstacle to printing output exists. 

(4) Output data to a selected printer 

(5) Issuance of the selected printer and a notifica- 
tion that the data has been output. 

35 

At step S116, the job found at step S1 15 is regis- 
tered. The above process is repeated until, at step S1 1 7, 
there are no jobs to be searched for At step S118 a 
check is instituted to determine whether or not the job 
40 can be performed. When the performance of the job is 
not possible, another job that must be executed is 
searched for At S1 1 9, the jobs that have been searched 
for are sequentially performed. 



Fig. 6 is a diagram for explaining an example where- 
in a remote controller (hereinafter referred to merely as 
a controller) is employed as a means for sending instruc- 
50 tions to various devices. 

A controller 605, which has a display, displays a 
control panel for, or information concerning a device, 
such as a printer 601 or 602, a facsimile machine 603 
or a PC 604, only by approaching the device, and sends 
55 instructions to the device to control it. Further, the con- 
troller 605 can control a remote device across a network 
via the device that is near the controller 605. 

The controller 605 can also monitor and display the 
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status of each device, and can acquire the status of a 
remote device on the network via a device that is near 
the controller 605. 

[Third Embodinnent] 

Fig. 7 is a diagram showing an example where a 
document is read by a scanner and is transmitted to a 
specific printer. 

When a document has been read by a scanner 701 , 
a user designates an addressee by voice, etc. , or a des- 
tination transmission is designated on a cover sheet, 
and is thus automatically determined. When a mono- 
chrome printer 702 is determined as an addressee, 

(1 ) if the addressee is not in the ready tor printing 
state, the document data is transmitted to another 
printer (e.g.. amonochrome printer 703) for printing; 

(2) if a printer designated as an addressee can not 
print the document because of the conditions spec- 
ified, such as color and paper size, data lor only for 
the pertinent page is transmitted to another printer 
(e.g., a color printer 704 if color printing is request- 
ed); 

(3) data for a document having a large number of 
pages are allocated to a plurality of printers to print; 
or 

(4) if a transfer destination is designated at the ad- 
dressee, the data is transferred to the transfer des- 
tination. 

[Fourth Embodiment] 

Fig. 8 is a diagram showing an example where a 
document is read by a scanner and is transmitted to a 
specific printer that the scanner can not manage be- 
cause it is in a different network domain, etc. 

When, a document is read by a scanner 701 in Aus- 
tin, a user designates an addressee by voice, etc. , or an 
addressee that is described using a character string or 
a bar code on a cover sheet is read and recognized/ 
interpreted to determine the addressee. As a result, a 
monochrome printer 702 in Tokyo is determined to be 
the addressee. After the document data are transmitted 
to the nnonochrome printer 702: 

(1) when the monochrome printer 702 is not ready 
for printing, it transmits the data to another printer 
(a monochrome printer 703) for printing; 

(2) if the monochrome printer 702 is not ready for 
printing the received document data because of 
specified conditions, such as color and paper size, 
it transmits either all the data for the received doc- 
ument or only data for pertinent pages to another 
printer (e.g., a color printer 704, if color printing is 
requested); 

(3) if a document has a large number of pages, the 
monochrome printer 702 prints part of the data and 



also allocates printing data to a plurality of other 
printers; or 

(4) if a transfer destination is designated at the mon- 
ochrome printer 702, the printer 702 transfers the 
5 data to the transfer destination. 

A difference between the third and the fourth em- 
bodiments is that in the third embodiment the scanner 
701 determines the addressee, and in the fourth embod- 
10 iment the printer 702 that received the document from 
the scanner 701 determines the transfer destination. 

[Fifth Embodiment] 

15 Fig. 9 is a diagram showing an example wherein a 
document is read by a scanner and is transmitted to a 
specific printer that the scanner can not manage be- 
cause it is In a different network domain, etc. 

When, a document is read by a scanner 701 in Aus- 
20 tin, a user designates an addressee by voice or using a 
keyboard, or an addressee that is described using a 
character string or a bar code on a cover sheet is read 
and recognized/interpreted to determine the addressee. 
As a result, a monochrome printer 702 in Tokyo is de- 
25 termined as the addressee. After document data are 
transmitted to the monochrome printer 702, they are fur- 
ther transferred to a personal computer 705 that can 
perform a higher level operation. 

As a result, the personal computer 705 determines. 
30 from the entry "urgent" on the cover letter, for example, 
that the document is an urgent document. The personal 
computer 705 examines schedule data to find the loca- 
tion of the recipient of the document, and transfers the 
data to that location, for example, Atsugi. At this time, if 
35 the transfer destination is not a printer but is a different 
medium, such as a facsimile machine 706. medium con- 
version is performed before transmission, 

[Sixth Embodiment] 

40 

Fig. 10 is a diagram showing an example wherein 
a document is read by a scanner and is transmitted to 
a specific printer that the scanner can not manage be- 
cause it is in a different network domain, etc. 
45 When, a document is read by a scanner 701 in Aus- 
tin, a user designates an addressee by voice, etc., or an 
addressee that is described using a character string or 
a bar code on a cover sheet is read and recognized/ 
interpreted to determine the addressee. As a result, a 
50 monochrome printer 702 in Tokyo is determined as the 
addressee. After document data are transmitted to the 
monochrome printer 702, they are further transferred to 
a personal computer 705 that can perform a higher level 
operation. As a result, the personal computer 705 iden- 
55 tifies the recipient, prepares a notification sentence, and 
notifiesthe recipient of the arrival of the document. 
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[Seventh Embodiment] 

Fig. 11 is a diagram showing an example wherein 
a notification concerning the status of a printer is trans- 
mitted. 

When a PC 1102 acquires the status of a printer 
1103 across a network and detects a change in the sta- 
tus, such as the occurrence of an error or an event, the 
PC 1102 determines a notification destination, such as 
a user or a manager, in consonance with the contents 
of the change, and employs a medium corresponding to 
the determined destination to transmit a notificatbn. For 
example, to transmit a notification to a terminal 1101 of 
a user or a manager, the PC 1102 uses electronic mail, 
or a telephone 1 1 04, or transmits a message to a pager 
1105. 

Therefore, in the example in Fig. 10, tor example, a 
notification of the reception of the document may be is- 
sued using electronic mail or a pager, instead of the tel- 
ephone 707. 

[Eighth Embodiment] 

In this embodiment, before executing an instructed 
job an apparatus notifies a user of the contents of the 
job in accordance with the type or the condition of the 
job, or cancels a job that is determined to be unneces- 
sary for a user. 

Fig. 1 2 is a flowchart showing the processing in this 
embodiment. Fig. 13 is a table showing, in consonance 
with the types and conditions of jobs, whether or not an 
advance notice should be issued. 

In Fig. 12, first, at step S120 a check is performed 
to determine wtiether or not a job constitutes necessary 
information for a user. If this is true, the processes at 
step SI 22 and the following steps are perfonmed. If not, 
at step 81 27 the job is canceled. At step SI 22, informa- 
tion concerning the advance notice that corresponds to 
the job type and conditions in Fig. 1 3 is referred to, or 
the contents described on a cover sheet are analyzed 
to determine whether or not a notice for the process to 
be executed and the contents of the process should be 
issued to a user prior to the perfomnance of the instruct- 
ed job. When advance notice is not required, program 
control moves to step SI 26. When advance notice is 
issued, program control moves to step SI 24, whereat a 
user is notified of the process to be perfomned and its 
contents. At step S125, the permission for the perform- 
ance is received from a user, if necessary. At step S1 26 
the job is performed. 

An explanation will be given for an example where- 
in, when the contents of a job call for the filing of infor- 
mation, an analysis of the system contents is performed 
prior to the filing of information, and information con- 
cerning the contents is transmitted to a user. 

Fig. 14A is a diagram showing an example wherein 
filing process information to be filed is transmitted to a 
user in advance. A file storage unit 1401, a scanner 



1402 and a user terminal 1403 are provided in this sys- 
tem. The processing performed by the scanner 1402 in 
Fig. 1 4A will now be described while referring to the flow- 
chart in Fig. 12. 

5 At step SI 20, since an instructed job is for informa- 
tion filing, it is assumed that this is necessary for a user. 
At step 8122, while information designated in Fig. 13 is 
referred to, it is ascertained that a notice concerning in- 
formation to be filed should be issued before the filing, 

10 and that the information should then be scanned. Pro- 
gram control then moves to step SI 24 to issue a notice 
concerning information to be filed. At step 8124 the 
scanned information is transmitted to the user terminal 

1403 via a network. At step SI 26, the filing job is per- 
^5 formed. 

Next, an explanation will be given for an example 
wherein the information on the first page is received and 
analyzed, and when the information is not at all related 
to a user, the following information is not received. 

20 Fig. 1 4B is a diagram showing an example wherein 
whether or not a large amount of data to be received is 
required by a user is determined by examining the first 
page, and the receipt of information for the following 
pages is canceled. A reception PC 1404 and a trans- 

25 mission PC 1405 are provided. 

The processing performed by the PC 1404 in Fig. 
14B will be explained while referring to the flowchart in 
Fig. 12. 

At step S120, the first page of the received data is 
30 analyzed. Since "For All Persons In Charge Of General 
Affairs Information" is entered on the first page, it is ap- 
parent that this document is not intended for a user who 
is not in charge of general affairs information, and the 
receipt of the information is determined to be unneces- . 
35 sary. Program control therefore moves to step SI 27. At 
step SI 27, the first page, which has been received, is 
canceled, and information for the second and the follow- 
ing pages is not received. The processing is thereafter 
terminated. 

40 

[Ninth Embodiment] 

Fig. 1 5 is a flowchart showing the processing for a 
ninth embodiment. In this embodiment, an apparatus 

45 that is designated to perform the processing determines 
whether it or another apparatus should perform the 
processing, and as a result of the determination, either 
performs the processing itself or sends an instruction to 
another apparatus to perform the processing. 

50 At step SI 50 a check is performed to determine 
whether or not a job has been input. If a job has been 
input, at step S1 51 the input job is entered in a job table. 
At step S152 the job table is examined to determine 
whether any jobs have been entered. If they have been, 

55 at step SI 53 a job is extracted from the table. At step 
S154 a received instruction is analyzed and the object 
of the job is apprehended using the result At step SI 55 
the status of the apparatus is examined. At step SI 56 
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the status of another apparatus consonant with the ob- 
ject of the job is examined 

When, as the result of a comparison of the statues 
of the locally owned apparatus and other apparatuses, 
it is found that the locally owned apparatus is optimal for 
the performance of the job, program control moves from 
step SI 57 to step SI 58, whereat it is determined that 
the owned apparatus will perform the job, and at step 
SI 59 the job is performed by the locally owned appara- 
tus. If there is an optimal apparatus other than the locally 
owned apparatus that has capabilities consonant with 
the object of the job, program control goes from step 
SI 60 to step S161 , whereat it is determined that the oth- 
er apparatus will perform the job, and at step SI 62 the 
locally owned apparatus sends a job execution instruc- 
tion to the apparatus having capabilities consonant with 
the object of the job. At step SI 63 the locally owned ap- 
paratus notifies a user that another apparatus has exe- 
cuted the job. 

If there is no apparatus having capabilities conso- 
nant with the object of the job, program control moves 
from step SI 60 to step S 1 64, whereat a plan is prepared 
for the use of an optimal method that does not depart 
from the object of the job, and at step SI 65 the plan is 
proposed to a user 

Fig. 16A is a diagram showing the transmission of 
information in this embodiment. 

When a user of a PC 101 provides for a printer 104 
a job for the output of information input by a scanner 
102, and when the printer 104 has, for example, a print- 
ing malfunction, the printer ascertains that it can not per- 
form the instructed job, and by communicating with oth- 
er apparatuses, it finds a printer 103 that can perform 
the instructed job. Thus, the printer 104 determines that 
the printer 103 can act for it and perform the job, and so 
instructs the printer 103. Further, the printer 104 trans- 
mits a notification to the PC 1 01 to inform the user of the 
action that it took. In Fig. 1 6A, the process flow for these 
activities is described by arrows. 

The processes performed by the individual units in 
Fig. 16A will be described while referring to the flowchart 
in Fig. 15. 

First, the process performed by the scanner 1 02 will 
be explained. At step SI 50 the scanner 102 scans a 
document and acquires a job, information concerning 
which is to be transmined to the printer 104. At step 
S 1 51 this job is entered in the job table, and at step SI 53 
the job is extracted. At step S154, the scanner 102 
scans additional data and determines that the data is a 
job to be transmitted. At step SI 55, the scanner 1 02 as- 
certains that it is operating nonnally and that no problem 
exists in the scanning and transmission of instructed in- 
formation. 

At step SI 56 it is ascertained that the job can not 
be transmittedto another apparatus. At step SI 57, since 
optimally the scanner 102 should execute the job, pro- 
gram control moves to step SI 58. At step SI 58 the 
scanner 102 determines that it should scan the informa- 



tion, and at step SI 59 the scanner 102 scans the infor- 
mation and transmits it to the printer 104 via a route 1 
on a network. 

At step SI 50, the printer 104 receives the job for 

5 the output of received information. At step S151, the 
printer 104 enters the job in the job table, and at step 
SI 53, it extracts it. At step SI 54, the printer 104 ascer- 
tains from the information it receives that the information 
for the user from the PC 101 is to be printed on paper. 

10 At step SI 55 the printer 104 determines that it is out of 
toner and disables printing. At step S1 1 6 the printer 1 04 
searches, via the network, for printers that can output 
the received information, and finds the printer 1 03 (route 
2). 

is Since, at step SI 57, the printer 1 04 can not execute 
the job, program control moves to step SI 60. Then, as 
at step 8160 it is determined that the job for the output 
of infomnation can be performed by the printer 103, at 
step SI 61 it is determined that the received information 

20 should be transmitted to the printer 103. At step SI 62 
the printer 104 sends an instruction to the printer 103 to 
print the information that is to be received (route 3). At 
step SI 63 to notify the user that the instructed job was 
performed by the printer 1 03 electronic mail for the user 

25 is transmitted to the PC 101 (route 4). 

Fig. 16B is a diagram showing the transmission of 
information for a case where no apparatus having ca- 
pabilities consonant with the object of a job is available. 
When the user of the PC 1 01 provides a job for the 

30 output to the printer 1 04 of information input by the scan- 
ner 102, and when the printer 104 disables printing be- 
cause it is out of toner, the printer 104 communicates 
with other apparatuses, finds the printer 103, which is 
suitable for the object of the job, and determines its sta- 

35 tus. Then, since the printer 103 disables printing be- 
cause it is out of paper, and as there is no other printer 
available that is suitable for the object of the job, a notice 
is issued to the PC 101 to propose to the user an optimal 
plan whereby when either the printer 103 or 104 has re- 

40 covered to the printing enabled state, that printer will 
perform the job. In Fig. 16B, the process flow for these 
actions is indicated by arrows. 

The procedures performed by the individual units in 
Fig. 1 6B will be explained while referring to the flowchart 

45 in Fig. 15. 

The process whereby the scanner 1 02 reads a doc- 
ument and transmits it to the printer 104 is performed in 
the same manner as was described previously. 

At step SI 50 the printer 104 receives a job for the 

50 output of received information. At step SI 51, the job is 
entered in the job table, and at step SI 53, it is extracted. 
At step SI 54, the printer 104 ascertains from the infor- 
mation it receives that the information for the user from 
the PC 101 is to be printed on paper At step SI 55, the 

55 printer 1 0 ascertains that it is out of toner and printing 
is disabled. At step SI 56, the printer 104 communicates 
across the network with other printers to find one that 
can print the received information. The printer 104 de- 
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termines that the printer 103 has also disabled printing 
because it is out of paper, and that no other appropriate 
apparatus is available. 

Since, at step S157, the printer 104 disables the 
performance of the job, program control moves to step 
SI 60. At step S160, as there is no apparatus that can 
perform the job for the output of information, program 
control goes to step SI 64. At step SI 64, planning is be- 
gun to prepare an optimal method for performing the 
printing, which is the object of the job for the output of 
information As a result, it is determined that the optimal 
method involves the return of a printing apparatus to its 
normal operating status. At step SI 65, the proposed 
method, whereby either the printer 104, which is out of 
toner, or the printer 103. which is out of paper, is to be 
recovered to the oriniing enabled state, is presented in 
a window shown m f- ig 17 to^ approval by the user. 

Then, a response rcccr/oc from the user, or a 
change in the printer status is »dcnittied as an input job, 
and the same processing is agriin performed. In this 
fashion, the ob|ect ol the usoi c^n be achieved. 

The above described planning will be described lat- 
er in detail during the course of the explanation of spe- 
cific examples for an eleventh and subsequent embod- 
iments. 

[Tenth Embodiment) 

In the processing for the ninth embodiment, where- 
in, at steps S158 and S159. a locally owned apparatus 
determines to perform a job itself, and performs the job. 
before another instructed job is performed, the possibil- 
ity of the occurrence of a problem is analyzed in detail, 
as in the procedures in Fig. 17. When it is ascertained, 
in consonance with environmental conditions or the sta- 
tus of the job performance, that performing the job is not 
advisable, a notice concerning the problem may be is- 
sued, or the request for the performance of the job may 
be rejected- 

When, for example, an instruction for printing a se- 
cret document is issued and a user is not identified as 
an authorized user, the instruction is rejected. 

Fig. 18 is a flowchart showing the detailed proce- 
dures for the performance of a job. 

At step SI 80 a check is performed to determine 
whether or not a problem exists concerning the perform- 
ance of the instructed job. At step SI 81 no problem af- 
fecting the performance of a job is found, at step SI 85 
the job is performed. If a problem is found, at step S1 82 
a check is performed to determine whether a notice con- 
cerning the problem that was discovered should be 
transmitted to a user, or whether the request for the per- 
formance of the job should be rejected. When it is as- 
certained that such a notice should be transmitted to a 
user, at step S184 the notice is transmitted to the user. 
When it is determined that the request for the perform- 
ance of the job is to be rejected, at step SI 86 the request 
for the performance of the job is rejected and a notice 



of rejection is transmitted to the user 

An example wherein a user of the PC 101 issues 
an instruction to output to the printer 104 confidential 
information read by the scanner 102 will now be de- 
5 scribed while referring to Figs. 15 and 18. Fig. 19 is a 
diagram showing example confidential information to be 
read. 

First, at step SI 50 the scanner 102 scans available 
information and receives a job for the transmission of 

10 information to the printer 104. At step SI 51 the job is 
entered in the job table, and at step SI 53, the job is ex- 
tracted therefrom. At step SI 54 data are scanned, and 
it is ascertained the data constitute a job to be transmit- 
ted. At step S155, the scanner 102 determines that it is 

IS operating normally andthat there is no problem affecting 
the scanning and transmission of the instructed infor- 
mation. 

At step SI 55 it is ascertained that the job can not 
be transmitted to another apparatus. Since, at step 

20 S157, it is ascertained that optimally the scanner 102 
should perform the job itself, program control moves to 
step S158, whereat the scanner 102 determines that it 
should scan the information itself. 

At step SI 80, as a result of the scanning of the in- 

25 formation, it is determined to be confidential information 
whose output is inhibited. Since a problem affecting the 
performance of the job exists, at step S182 it is deter- 
mined that the job (the scanning and the outputting of 
information) should be rejected. At step SI 83 the job is 

30 rejected, and program control thereafter goes to step 
SI 86. At step SI 86, a notice is issued that the job was 
rejected. 

Similarly, when a plurality of users employ the same 
system, the current user can be identified by a log-in 

35 name, and an instruction for the output of the contents 
of files belonging to other users can be rejected. When, 
instead of an instruction from a user, a document is re- 
ceived from the outside and the destination user differs 
from the current user, the output of the received docu- 

40 ment and the notice of the receipt of the document are 
rejected, or the processing is halted until the destination 
user begins to employ the system. 

[Eleventh Embodiment] 

45 

The processing for an eleventh embodiment will 
now be described while referring to Fig. 4. At step SI 09 
the job table is initialized. Then, at step S110 a check is 
performed to determine whether there is input from a 

50 user or from an apparatus that can be detected by the 
system. At step S111 an input analysis job is entered in 
the job table to analyze the contents input at step S11 0. 
At step S 11 2 a check is performed to determine whether 
there is a job to be performed. If there is such a job, it 

55 can be acquired at step S113. At step S114, the object 
comprising the background for the performance of the 
job is understood. 

At step S1 1 5 a possible process is planned by using 
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the contents that are input, the status of a device that 
can be detected by the system, and the knowledge that 
the systenn has currently acquired. At step S116 a new 
job is added if necessary. At steps S1 17 to S119 the 
process as planned at step S1 1 5 is perlormed in conso- 
nance with the contents. The processing is repeated un- 
til no more jobs remain, and program control thereafter 
returns to step S110. 

A case where a user is to print <file A> will now be 
explained. Fig. 23 is a diagram showing the contents of 
<fjle A>, the colored picture of a car. Fig. 1 is a diagram 
showing the structure of the system in this embodiment. 
A user sends an instruction from a PC 1 01 , and a mon- 
ochrome printer 104 is set as a normally used printer. A 
color printer 1 03 is also set. 

At step S109 the job table is initialized. AtstepSUl 
a job that is to be performed when there is no input is 
added to the job table (Fig. 20). When file name <file A> 
is designated in a window in Fig. 21 and printing for it is 
selected, at step 8110 it is determined that there is input, 
and program control advances to step 8111. At step 
8111, as is shown in Fig. 22, an input analysis job is 
entered in the job table. Since, at step S112, there is a 
job to be executed, program control advances to step 
S113. At step S113 an analysis of the job to be per- 
formed (input "print <file A>'') is acquired. At step S1 14 
it is ascertained that the object of the input "print <file 
A>'' is the printing of the <file A>. 

At step S1 15 planning is performed, based on the 
following conditions and statuses, for proposing, to a us- 
er that the printing be performed by a color printer. 

* <file A> includes a colored portion (the portion com- 
prising the car in Fig. 23) 

* the color printer 103 that can be used is present 
(Fig. 1). 

At step S119 a query "Perform printing by a color 
printer?' is directed to a user (Fig. 24). At the same time, 
a job corresponding to an event is added to the job table 
when no response is received from the user (Fig. 25). 
Program control thereafter returns to step 8110. 

When, at step 8110, there is no input for ten min- 
utes, at step S1 1 2 it is assumed that there Is a job to be 
performed, and program control advances to step S113. 
At step 8113, a job to be performed (when no response 
is received from a user) is acquired. 

At step 8114 it is ascertained that the object for a 
job when there is no response from a user is to deter- 
mine whether a proposal can be accepted. At step S1 1 5 
it is determined from the flow condition and the status 
that the user should be queried by telephone. 

* The telephone number of the current location of the 
user is acquired (schedule data in Fig. 26) 

* The printing priority is urgent (according to the 
schedule data in Fig. 26, file A is to be used at the 
business talk at 1 3:00). 



At step S119 a query, "Perform printing with a color 
printer?" is directed to the user (Fig. 27). At the same 
time, a job corresponding to an event is added to the job 
table when no response is received from the user (Fig. 
s 28). Program control thereafter returns to step S11 0. 

When, at step 811 0, a user responds with "Yes, 
please" as is shown in Fig. 27, it is assumed that data 
input is to be performed, and program control moves to 
step S111 . At step 8111 an input analysis job is added 
10 tothe job table (Fig. 21). At step 811 2 it is assumed that 
there is a job to be performed, and program control 
moves tostep 8113. At step 8113, a job to be performed, 
the input, "Yes, please" is analyzed. At step S1 14 it is 
ascertained that the object of the input, "Yes, please" Is 
is the acceptance of the proposal. 

At step S1 15 it is determined that <fite A> is to be 
transmitted to and printed by the color printer 103. At 
step S1 18 <ffle A> is transmitted to and printed by the 
color printer 103. At the same time, a job for confirming 
20 whether or not the printing has been terminated normal- 
ly is added to the job table (Fig. 29). Program control 
thereafter returns to step 8110. 

If, at step 8110 there is no data input and at step 
8112, as is shown in Fig. 30, the printer status is 
25 changed from "Printing" to "Printed document in tray" it 
is assumed that there is a job to be executed, and pro- 
gram control advances to step 8113. At step S113 a 
process corresponding to the job to be executed, printer 
status, is acquired. At step 8114, from a pr'mter status 
30 entry in Fig. 31, the status "Printed document in tray" 
means that there is a printed document in a printer tray, 
and the object of the job is understood to be the delivery 
of the printed document to a user. 

At step 8115, a waiting state is specified because . 
35 the user may soon come to pick up the printed docu- 
ment. At step 8116, a job for confirming whether the user 
has come and has picked up the printed document is 
added to the job table (Fig. 32). Program control there- 
after returns to step 8110. If. at step 8110, there is no 
40 input for ten minutes and at step 8112 the printed doc- 
ument is still on tray, there is a job to be executed, and 
program control therefore advances to step 8113. At 
step S11 3 a process corresponding to the job to be ex- 
ecuted, when the printed document is still on hand, is 
45 acquired. At step Si 14, from the table in Fig. 31 the sta- 
tus "Printed document on tray" means that a printed doc- 
ument is still in the printer tray, and it is assumed that 
the object is the delivery of the printed document to the 
user. Since at step 8115 the printed document has not 
50 been picked up for ten minutes, it Is assumed that the 
user has forgotten to collect the document, and It is de- 
termined that a notice, "Printed document is in color 
printer tray" is to be issued (Fig. 33). Also, a job corre- 
sponding to the failure of the user to respond and a job 
55 corresponding to the collection of the printed document 
by the user are added to the job table (Fig. 34). Program 
control thereafter returns to step 811 0. 

When at step S1 10 there is no input and at step 
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S112 the printer status is changed Irom "Printed docu- 
ment in tray" to "Normal" as is shown in Fig. 28, there 
is a job to be executed and program control advances 
to step S1 13. At step S113 a job corresponding to the 
job to be executed, printed document in tray, is re-eval- 
uated. At step S114 the object for the rechecking of the 
job corresponding to the "Printed document in tray" is 
understood to be the deletion of an unnecessary job. At 
step S115, it is determined that the job of waiting for a 
response from the user should be deleted, and at step 
S1 1 9 the job of waiting for the user's response is delet- 
ed. A new job is not added. While the job table is in its 
initial state, program control returns to step 3110. 

Figs. 36 and 37 are tables showing plans, actions 
and rules. 

When a user presses the button "Print" in the print- 
ing window in Fig. 21 , it is determined by referring to the 
plans and actions in Fig. 36 that the object of the user 
is the changing of the current state to the "Printed doc- 
ument in tray". In addition, since "Printed document in 
tray" is the assumption for the object "Acquire printed 
document" the final object of the user is determined to 
be "Acquire printed document". 

When a user expresses agreement or disagree- 
ment by voice or by the manipulatbn of the mouse in 
response to the query in the inquiry window in Fig. 24 
or the window in Fig. 33, or the telephone query in Fig. 
27, it is determined that the object of the user is the state- 
ment of the agreement or disagreement relative to the 
query. 

In Fig. 37 the final object of the system is the thus 
constitutes the achievement of the object of the user As 
the basis for this, the system must be stable. In addition, 
the system must (1) apprehend the object of the user 
and (2) must make a plan for the achievement of the 
object and execute it. 

To stabilize the system, not only an abnormal timing 
for the system must be adjusted to the normal condition, 
but also the system must be powered off when it is not 
required in order to reduce costs and to prevent the sys- 
tem from becoming unstable. 

Fig. 38 is a diagram that for easier understanding 
shows only one part of the table in Fig. 37. 

In order to understand the object of the user, data 
that is input is identified. Further, corresponding to "ac- 
quire printed document" which is the object of the user 
in Fig. 36, "Transfer printed document to user" which is 
the object of the system. 

The presence of a printed document is the premise 
for the transfer of the printed document to a user, and 
the object is achieved by notifying the user of the loca- 
tion of the document. Of course, the user must collect 
the document or the document must be delivered to the 
user by some means in order for the user to acquire the 
document; in this embodiment, it is determined that 
when the printed document is no longer in the printer 
tray, that is the equivalent of the user having acquired 
the document. 



The presence of an original document and the print- 
ing conditions are required in order to obtain a printed 
document, and it is premised that a corresponding print- 
ing environment normally exists. In this situation, a cor- 

5 responding printing environment is selected to perform 
printing. The original document must be so prepared 
that its existence is constant. In addition, to clarify the 
printing conditions, only when there is a question con- 
cerning a condition need ask a user queries. When a 

10 printing environment is abnormal, the printing environ- 
ment is normalized to maintain normal environment. 
When no response is received from a user within a pre- 
determined period of time, additional planning must be 
performed to ascertain the intent of the user. 

15 The properties of each device are stored in a mem- 
ory in each device, or in the memory of the server that 
manages each device. Fig. 39 is a table showing the 
properties of the devices. The statuses of the devices 
are held in the devices, and are voluntarily transmitted 

20 lo other apparatuses by the devices, or by the passive 
transfer of them when requested by another server. 

In this system, the properties are inherent to the in- 
dividual devices and do not change; they are acquired 
when the system is activated or when a connection 

25 across the network is altered. In the planning process, 
the statuses are acquired, as needed, or at a specific 
time interval, and are updated as information intended 
for the internal memory. 

30 [Twelfth Embodiment] 

An explanation will now be given for the processing 
performed in a case where, as in the eleventh embodi- 
ment, a color printer is busy when <file A> is to be print- 
35 ed. 

First, when "Pnnt <file A>'' is input, it is entered in 
the job table, and its object is understood lo be a request 
that printing be performed. Assume as the condition/sit- 
uation that the colored portion is included in a document 

40 and a color printer is busy A plan is devised to query a 
user concerning whether the job can wait until the color 
printer is not busy, or whether the data can be printed 
in monochrome. The query. "The color printer is busy. 
Wait, or print in monochrome?" is presented to a user. 

45 When the user selects "Wait" the process is placed on 
standby until the color printer is no longer busy. 

[Thirteenth Embodiment] 

50 When "<file A> was changed to <file A'>" is input, it 
is ascertained that the updating of the job table is the 
object. As the condition/situation, the <file A> printing 
job is stored in the job table. Thus, a plan is made to 
query a user concerning the changing of the printing tar- 

55 get to <file A*>. Then, the query "Print <file A'> instead 
of <file A> before amended?" is presented to the user 
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[Fourteenth Embodiment] 

According to this embodiment, in a system wherein 
a plurality of apparatuses are connected together (in this 
case, they are connected across a network), when a job 
tor an individual apparatus is instructed, an analysts of 
the object of the job is performed by the appearance. 
When the apparatuses determine from their statuses 
and the status of the instructed apparatus, that the in- 
structed apparatus should not execute the job, the per- 
formance ol the job is transferred to another apparatus. 

Figs. 41 A and 41 B are diagrams in each of which 
is shown the condition where a plurality of apparatus are 
connected lo a network. A network in Fig. 41 A provides 
an environment v^itterein a scanner 411 and printers 41 2 
and 413 can determine the states of the other appara- 
tuses. A network in Fig 41 B provides an environment 
wherein the scanner 411 and the pnniers 412 and 413 
can not determine ihc states ol the other apparatuses. 

Fig. 42 is a diagram illuslfaling a lunclional arrange- 
ment tor pertorming !he pioccbsiay m xhis embodiment. 
A job reception unit 421 receives a job from a user or 
from another apparatus A received job is registered in 
a job table 422. A job analysis unit 423 extracts the job 
from the job table 422 and analyzes it. Another appara- 
tus information acquisition unit 424 determines the at- 
tributes and the current statuses of other connected ap- 
paratuses. A self state determination unit 427 deter- 
mines the attribute and the current state of the self ap- 
paratus. 

An optimum planning unit 425 prepares an optimal 
plan for the performance of a job, A job performance 
determination unit 428 determines whether or not a job 
is to be performed, and whether the locally owned ap- 
paratus or another apparatus should perform the job. 
When the job is to be performed by the locally owned 
apparatus, a job performance unit 426 performs it. 
When another apparatus is to perform the job, another 
apparatus job transfer unit 429 transfers the job to an- 
other apparatus. A transfer notification unit 430 notifies 
a user of the results obtained by the job performance or 
the transfer of the job to another apparatus. Fig. 43 is a 
main flowchart showing the processing performed for 
this embodiment- 

At step S430, in order to determine whether a job 
has been input, a check is performed to determine 
whether or not a job has been input by a user, a job has 
been input by a device that can be detected by a system, 
based on the results obtained by analyzation of the jobs, 
or a new job that has been generated by the locally 
owned apparatus while idling. When a job has been in- 
put, at step S431 , an input job is added to the job table. 
At step S432, the job table is examined to determine 
whether there is a job. If there is a job, at step S433 the 
job is extracted. At step S434 a received instruction is 
analyzed, and from the information attained by the an- 
alyzation, the object of the job is obtained. At step S435 
a check is performed to determine whether or not a job 



for another apparatus es present. If there is such a job, 
at step S436 control of another apparatus is initiated. If 
there is no such job, at step S437 a control process the 
locally owned apparatus is initiated. 
5 Fig. 44 is a flowchart for the process for controlling 
another apparatus performed at step S436 in the main 
flowchart. 

At step S440 the status of an apparatus that is des- 
ignated in the job is examined, and at step S441 , based 

10 on the status, a check is performed to determine wheth- 
er a problem affecting the performance of the job by the 
designated apparatus exists. If no such problem exists, 
at step S442 the job is transmitted to the designated ap- 
paratus. If a problem affecting the performance of the 

15 job by the designated apparatus exists, at step S443 a 
check is performed to determine whether or not there is 
an apparatus other than the designated apparatus that 
is suitable for the performance of the job. It there is such 
an apparatus, its status is examined. As a result, if there 

20 is another apparatus that can execute the job, at step 

5445 the job is transferred to that apparatus, and at step 

5446 the user is notified of that the job was transferred 
to that apparatus. When there is no apparatus otherthan 
.the designated apparatus that can perform the job, or - 

25 when another suitable apparatus can not be used at that 
time, at step S447 an optimal plan is devised that will 
not degrade the object of the job, and at step S448, the 
plan is proposed to the user 

Fig. 45 is a flowchart for the process for controlling 
30 the locally owned apparatus performed at step S437 in 
the main flowchart. 

At step S450 the status of the locally owned appa- 
ratus is examined, and at step S451 , based on the sta- 
tus, a check is performed to determine whether a prob- 
35 lem affecting the perfomnance of the job by the locally 
owned apparatus exists. If no such problem exists, at 
step S452 the job is performed by the locally owned ap- 
paratus. If a problem affecting the performance of the 
job by the locally owned apparatus exits, at step S453 
40 a check is perfomned to determine whether or not there 
is an apparatus other than the locally owned apparatus 
that is suitable for the performance of the job. If there is 
such an apparatus, its status is examined. As a result, 
if there is another apparatus that can execute the job, 
45 at step S455 the job is transferred to that apparatus, and 
at step S456 the user is notified that the job was trans- 
ferred to that apparatus. When there is no apparatus 
other than the locally owned apparatus that can perform 
the job, or when another suitable apparatus can not be 
so used at that time, at step S457 an optimal plan is devised 
that will not degrade the object of the job, and at step 
S458, the plan is proposed to the user 

Fig. 46 is a diagram showing the condition that ex- 
ists when, while, in Fig. 41 A, an instruction is issued to 
55 output to the printer 412 along route A information that 
has been read by the scanner 411, the other apparatus 
information acquisition unit 424 of the scanner 411 dis- 
covers a malfunction at the printer 412. and this infor- 
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mation is output to the printer 413 along route B. 

Fig. 47 is a diagrarri showing the condition that ex- 
ists when, while, in Fig. 41 B, the scanner 411 is instruct- 
ed to output the information that is read to the printer 
412 along route A and to confirm the status ot the printer 
412, since the printer 412 is located in a remote area, 
the scanner 411 takes the status of the network into ac- 
count and unconditionally transmits the information to 
the printer 412, which then, because a malfunction has 
occurred there, transfers the received information (job) 
to the printer 41 3. 

The processing performed in Figs. 46 and 47 wil! be 
described while referring to the flowcharts shown in 
Figs 4310 45 

When m Fig 46 a job is to be output by the scanner 
411 and transferred to the printer 412. the scanner 411 
performs the tollowmg process. 

First at slop S430 the scanner 411 receives a job 
lor the scanning of information and the transmission of 
It lo Uie pnnier 4t2 Al step S431 the job is entered in 
ifie |Ob Irible ai^Uat step S433, it is extracted therefrom. 
Al step S434 the scanner 411 scans the data and as- 
ccnains the |Cb involves the transmission of data to a 
designated pnntcr At step S435 it is determined that 
there is a job which is to bo executed by another appa- 
ratus, lor the output ol the scanned information by the 
printer 412 Based on this determination, at step S436 
program control moves to another apparatus control 
process in Fig 44 

Al step 5440 the current state of the printer 412 is 
acquired. Then, information that the printer 41 2 has mal- 
functioned IS received, and at step S441 it is determined 
that printing by the printer 41 2 is disabled. In this case, 
besides a malfunction, the printer 41 2 can also be in an 
out of paper or out of toner disabled state. At step S443, 
it Is assumed that the designated printer 412 can not 
execute the job. and another apparatus that can execute 
the job is searched for by communicating with the other 
apparatuses across the network. At step S444 it is found 
that the printer 413 can execute the job and the perform- 
ance of the job is assigned to It. 

At step S445 the scanned information is transmitted 
to the printer 413. And at step S446, since a process 
other than the one instructed is being performed, the us- 
er is notified of the job alteration. Since the process for 
the instructed job has been terminated, the system waits 
for the next job. 

Following this, a process in Fig. 47 where a job is 
to be output by the scanner 411 and transmitted to the 
printer 412 will now be explained. 

First, at step S430 the scanner 411 receives a job 
for the scanning of information and the transmission of 
the information to the printer 412. At step S431 the job 
is entered in the job table, and at step S433, it is extract- 
ed therefrom. At step S434 the scanner 411 scans the 
data and ascertains the job is for the transmission of 
data to a designated printer. At step S435 it is deter- 
mined that there is a job, which is to be executed by 



another apparatus, for the output of the scanned infor- 
mation by the printer 412. Based on this determination, 
at step S436 program control moves to another appa- 
ratus control process in Fig. 44. 

5 At step S440, while the scanner 41 1 communicates 

with the printer 412 and attempts to acquire its status, 
it is found that the printer 412 is in a remote area and 
the scanner 411 decides to transmit the information, re- 
gardless of the status of the printer 412. At step S441 it 

10 is determined that there is no problem with the printer 
41 2 since it is in a remote area, and at step S442 the 
scanned information is transmitted to the printer 412, 
which is the designated apparatus, and the job is trans- 
ferred thereto. Since the instructed job has been termi- 

is nated, the scanner 411 waits for the next job. 

At step S430 the printer 41 2 receives the job tor out- 
putting the received information. At step 8431 the job is 
entered in the job table, and at step S433, it is extracted 
therefrom. At step S434 the printer 41 2 understands that 

20 the job is for the printer 41 2 to output the received infor- 
mation. At step S435 it is determined that there is no job 
for another apparatus, and at step S437 program control 
moves to an own apparatus control process in Fig. 45. 
At step S450 the current state of the printer 412 Is 

25 examined, and at step S451 the printer 412 judges by 
itself that the printer 412 has a malfunction and is In a 
printing disabled state. In this case, besides the mal- 
function, the printer 412 can be in such an output disa- 
bled state as out of paper or out of toner. At step S453, 

30 another apparatus that can execute the job is searched 
for by communication with the other apparatuses across 
the network. 

At step S454 the printer 413 that can execute the 
job is found, and the execution of the job at the printer 
35 41 3 Is detemnlned- At step S455, the printer 412 trans- 
mits the received information to the printer 41 3. At step 
8456, since the process other than being instructed Is 
performed, the user is notified of the alteration of the 
job. In this embodiment, such a notice is issued to the 
40 instruction source user. Since the output destination is 
far away from the instruction source, the notice may be 
issued to a recipient. Since the instructed job has been 
terminated, the system waits for the next job. As at the 
printer 41 3 there is no processing problem and the prim- 
es er 41 3 can perform the job for the output of the received 
information, the printer 413 performs the job. 

In the above embodiment, the individual units auto- 
matically transfer the job. If there is a problem at each 
apparatus, a user may provide a transmission order for 
50 the transfer of the job by the apparatuses. In this em- 
bodiment, a user Inputs as a job the employment of a 
designated printer to print a scanned document; how- 
ever, a second and a third printer may at the same time 
be set for the job, so that if the designated printer mal- 
55 functions they can replace It and perfonn the printing. 
Thus, at step S443 or S453, the second or the third print- 
er can be selected as a suitable apparatus for the job 
and its status is examined. A second or a third printer 
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may be set as a printer to receive a job and to replace 
a designated printer when it is in a printing disabled 
state. However, even when a second or the third printer 
is not set for the job, the printer that receives the job can 
transfer it in consonance with destinations that are set 
internally. 

In addition to a malfunction and an operation disa- 
bled state, the following conditions can cause the per- 
formance of a lob to be canceled: the specifications may 
be inappropriate for the performance of the job, e.g., a 
case where a job for the printing of color data is issued 
to a monochrome printer; or immediate processing of 
the job may not be possible because there are too many 
jobs in a queue. 

[Fifteenth Embodinnent] 



In this embodiment, in a process for determining the 
status of the locally owned apparatus at step S450 in 
Fig. 45, schedule information for a user which is the ob- 
ject, is examined as a factor that is used to determine 
whether a problem exists that will affect the performance 
of a job involving the output to a user of urgent informa- 
tion received by a printer. 

An explanation will be given for a process that is to 
be performed in the environment shown in Fig. 48, 
where a user, to whom urgent information is to be trans- 
mitted, is currently at an outside location at which a print- 
er 412 is available that is connected to a network. 

In the procedures in Fig. 43. at step S434 it is as- 
certained that the object of the job is the transmission 
to a user of urgent information that has been received. 
At step S435, since at this time there is no job for another 
apparatus, program control moves to step S437. At step 
S437 the process advances to the locally owned appa- 
ratus control process in Fig. 45. 

At step S450, in order to determine the status of the 
locally owned apparatus, the schedule of a user who is 
designated as a transmission source is confirmed by the 
PC 41 4. and rt is ascertained that the user is at an out- 
side location. Since the user is at an outside location, at 
step S451 it is determined that a problem related to mak- 
ing urgent contact exists at the locally owned apparatus. 
At step S453 a check is performed to determine whether 
or not a printer that can output information is present at 
the user's destination. At step S454 it is confirmed that 
there is a printer 412 that can output information, and it 
is determined that the printer 412 will be used to output 
the information. At step S455 the information is trans- 
mitted to the printer 412. At step S456 the users at the 
transmission source and at the reception destination are 
notified that the job has been transferred. 

When, at step S443, there is no printer available to 
which to transfer the job but there is another apparatus 
that can be employed by the user, such as a PC, a fac- 
simile machine or a telephone, at step S444 it may be 
determined to perform the job by using one of the avail- 
able apparatuses. Since the information can not be 



transmitted unchanged, at step S445 the information to 
be output to the printer is converted into an electronic 
mail document or a facsimile document, or is changed 
to voice information for the output. 
5 At step S446, depending on the medium used for 
the output, a notice that the job was transferred to an- 
other medium need not be issued, and can be transmit- 
ted by using the same medium. For example, when a 
telephone is used to transmit the information, it is better 
10 to notify a user of the job transfer before or after the 
transmission of the information. 

Instead of the telephone, at step S443, the above 
PC, the facsimile machine or the telephone may be de- 
termined to be a suitable apparatus to replace the printer 
15 for the job. At step S447 the employment of such an ap- 
paratus may be determined to be an optimal plan that 
does not degrade the object of the job, and at step 8448 
the plan may be proposed to a user 



20 [Sixteenth Embodiment] 

Fig. 49 is a flowchart showing the processing for this 
embodiment. Steps 3498 to S500 are additionally pro- 
vided for the flowchart in Fig. 43 as a process to be per- 
2S formed when there is no job. At step S498 the current 
operating statuses of other apparatuses are examined, 
and at step S499 a check is performed to determine 
whether or not a job that the locally owned apparatus 
can perform is being held in another apparatus and not 
30 being performed. If such a job is being held in another 
apparatus, at step 8500 the job is extracted from the 
pertinent apparatus and is entered in the job table for 
the locally owned apparatus, while it is deleted from the 
job table for the pertinent apparatus. In this embodi- 
35 ment. therefore, an apparatus finds a job by itself and 
processes it. 

Fig. 50 is a conceptual diagram showing the situa- 
tion for this processing- Since no job is present in a print- 
er 41 2, and a printer 41 3 has ten jobs in a queue, a print- 
40 er 4 1 5 acquires a job from the printer 41 3 and processes 
it. 

The process performed by the printer 415 ts as fol- 
lows At step S492 there is no job, and program control 
moves to step S49B. At step S498 the current statuses 
45 of the other printers, 41 2 and 41 3, are examined. At step 
S499 it is determined that at the printer 413 there is a 
job that the printer 415 can execute. At step S500 the 
printer 415 extracts that job from the printer 413, and 
enters it in its job table. Then, the extracted job is deleted 
50 from the job table for the printer 41 3. Since at step S490 
there is no job input, and at step S492 there is a job that 
was entered previously, program control moves to step 
S493 to execute the job. 

The operations for the perfomnance of the job by the 
55 locally owned apparatus, which were explained in the 
fourteenth and the fifteenth embodiments, are per- 
formed, and the processing is thereafter terminated. 
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[Seventeenth Ennbodinnent] 

Fig. 51 is a diagram showing the transmission in this 
embodiment of a print job and other information in a sys- 
tem employing apparatuses. The heavy arrows are 
used to depict the transmission routes tor the print jobs, 
and the broken line arrows are sued to depict the trans- 
mission routes for other information. 

The print job in this embodiment includes parame- 
ters that are referred to during printing: the document to 
be printed; the number of print copies; the print quality; 
and the print size. The other information includes the 
statuses, such as printing, normal, or out of paper, of 
individual printers, and a schedule for a print job proc- 
essed by a print job scheduling unit. 

In Fig. 51 , in a client machine 510, a print job gen- 
eration unit 512 generates a print job, and stores it in a 
print job memory unit 51 3. Then, the print job is trans- 
mitted by a print job transmission unit 514 in the client 
machine 510 to a server machine 511 for managing 
printers. 

In the server machine 511, a print job reception unit 
516 receives the print job. When the parameters of the 
print job have not yet been set, the print job is transmit- 
ted to a print job automatic setup unit 51 7, and when the 
parameters have already been set, the print job is trans- 
mitted to a print job automatic change unit 518. 

The print job automatic setup unit 517 sets print pa- 
rameters while taking into consideration the urgency, 
economy and the quality that are acquired by the print 
job. The print job automatic change unit 5 1 8 can perform 
the same process for changing parameters that have 
previously been set. The print jobs that are set or altered 
are registered in a print job schedule included in a print 
job scheduling unit 520 and are sequentially read to a 
print job transmission unit 519, or the print job may be 
directly transmitted to the printer job transmission on the 
unit 519 from the print job automatic setup unit 517 or 
the print job automatic change unit 518. The printing is 
then performed. 

The status, such as in printing, normal or out of pa- 
per, of an available printer is acquired by a printer state 
acknowledgement unit 515, and is transmitted to the 
print job automatic setup unit 517 and the print job au- 
tomatic change unit 518 where it is employed for the set- 
up or change of the print job. Similarly, the printing 
schedule for the print job included in the print job sched- 
uling unit 520 is also transmitted to these units 517 and 
518 for their employment. 

As is shown in Fig, 51 , a print job may be transmitted 
along a different route. When a machine for generating 
a print job is the same as a machine for managing print- 
ers, the print job may not be transmitted via the print job 
transmission unit 514. 

Fig. 52 is a flowchart showing the processing per- 
formed by the print job automatic setup unit 517. The 
print job automatic setup unit 517 automatically sets pa- 
rameters within a range that satisfies a designated qual- 



ity and the printing can be performed as rapidly and as 
economically as possible. 

In Fig. 52, at step S520 a factor for determining the 
printing time and the amount of consumed toner is ini- 
5 tialized in preparing for the simulation. Then, at step 
8521 a quality determination variable Is initialized to a 
maximum quality value (e.g., 5), and at step S522 the 
period of time for the printing is simulated. At step S523 
a check is performed to determine whether the simulat- 
ed ed period of time for the printing equals a specified val- 
ue. When the time period for the printing equals a spec- 
ified value, at step 8524 the consumption of toner is sim- 
ulated. At step 8525 a check is performed to determine 
whether the simulated consumption of toner equals a 
15 specified value. When the consumption of toner equals 
a specified value, the job is changed to the current qual- 
ity determination variable value at step S526. 

When, at step 8523 or 8525, the period of time for 
the printing or the consumption of toner is not equivalent 
20 lo its specified value, at step 3527 the quality determi- 
nation variable is decremented by one. At step 8528 a 
check is performed to determine whether or not the re- 
sultant quality determination variable satisfies is equiv- 
alent to a designated quality. When the designated qual- 
ms ity is satisfied, program control returns to step 8522. 
When the designated quality is not satisfied, the 
processing is terminated with an automatic setup disa- 
bled state. 

Fig. 53 is a table showing the period of time for the 
30 printing and the consumption of toner for one character 
corresponding to different specified qualities for the 
printers A521 and B522. These values may be fixed val- 
ues inherent to individual printers, or may be dynamic 
variable values, which are averages obtained for print- 
35 ing periods or for consumption of toner that were meas- 
ured in the past. 

Fig. 54 is a diagram showing the contents of a print 
job that is to be output in this embodiment, and param- 
eters that were designated by a user. In the system for 
40 the embodiment, the contents of a print job in Fig. 54 
and the set-up parameters are collectively called a print 
job. In this embodiment, calculated in terms of charac- 
ters, 500 characters are employed as the printing quan- 
tity for the print job in order to simulate the period of time 
45 tor the printing and the consumption of toner. In addition, 
it is assumed that for urgency a user will designate a 
period of only one minute, and will keep the default val- 
ues for economy and quality. 

In the example shown in Fig. 54, when the print job 
50 automatic setup unit 517 is activated, at step S520 in 
preparing of simulation, a factor for determining the pe- 
riod of time for the printing and the consumption of toner 
is initialized. As defined in the table in Fig. 53, for exam- 
ple, the periods of time required for the printing of one 
55 minimum quality (= 1 ) character by the printer A521 and 
the printer B522 are, on average. 100 mS and 200 mS 
respectively, and the consumption of toner for one char- 
acter are 100 mg and 200 mg. 
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Following this, at step S521 the quality determina- 
tion variable is initialized to the maximum quality value 
(e.g.. 5), and at step S522 the printing period of time for 
the printing is simulated. If, for example, the printers 
A521 and B522 are prepared tor printing and no print 
job is scheduled in the print job scheduling unit 520, 
printing can be initiated immediately and no waiting time 
is required. Thus, when the target print job includes 500 
characters, as simulation, printer A takes 250 seconds 
to print the characters and printer B takes 500 seconds. 

Then, at step S523 it is determined that this condi- 
tion does not satisfy the one minute urgency period that 
is specified by the parameter in Fig. 54, and at step S527 
the quality determination variable value is decremented 
by one 

Since the quality is not specified in the parameter 
in Fig 54 program control returns to step S522 lor sim- 
ulation ol the period of time for the printing. When, as 
the result of inc repetition of the above process, the 
quality is sol to the minimum quality ot 1, the period of 
tune lo. tt»e punting by the printer A521 is 50 seconds, 
whtch satisfies the urgent figure. Therefore, the quality 
IS set to the mmmum quality determination variable val- 
ue 1) and the processing is normally terminated. 

If the Guainy iG sot to 2 or higher, it does not meet 
the paramoiors sol by the user, and the processing is 
terminated as an automatic setup disable error. 

[Eighteenth Embodiment] 



Fig 55 IS a flowchart showing the processing per- 
formed by a print |Ob automatic change unit 518. The 
print job automatic change unit 518 automatically 
changes a parameter within a range that satisfies a 
specified urgency and economy so as to print data hav- 
ing as high a quality as possible. When all the specified 
parameters can not be satisfied, the most important des- 
ignated parameter is satisfied and the others are 
changed as slightly as possible. 

Fig. 56 is a diagram showing the contents of a print 
job that is to be output in this embodiment, and param- 
eters designated by a user. In the system of the embod- 
iment, the contents of a print job in Fig. 56 and the set- 
up parameters are collectively called a print job. In this 
embodiment, calculated in terms of characters, 500 
characters are employed as the printing quantity for the 
print job in order to simulate the period of time for the 
printing and Ihe consumption of toner. In addition, it is 
assumed that a user designates all of the parameters 
and particularly specifies that the quality is the most im- 
portant. 

In the example shown in Fig. 55, when the print job 
automatic change unit 518 is activated, at step S520 in 
preparing for the simulation, a factor for determining the 
period of time for the printing and the consumption of 
toner is initialized. As is defined in the table in Fig. 53, 
tor example, the period of time required for the printing 
of one minimum quality (= 1) character by the printer 



A521 and the printer B522 are, on average, 100 mS and 
200 mS respectively, and the consumption of toner for 
one character are 100 mg and 200 mg. 

Following this, at step S551 the quality determina- 
5 tion variable is initialized to the maximum quality value 
(e.g., 5), and at step S552 the period of time for the print- 
ing is simulated. If, for example, the printers A521 and 
B522 are prepared for printing and no print job is sched- 
uled in the print job scheduling unit 520, printing can be 
10 initiated immediately, and no waiting time is required. 
Thus, when the target print job includes 500 characters, 
as simulation, printer A takes 250 seconds to print the 
characters and printer B takes 500 seconds. 

Then, at step S553 it is determined that this condi- 
15 tion does not satisfy the one minute urgency period that 
is specified by the parameter in Fig. 56, and at step S556 
the quality determination variable value is decremented 
by one. 

Since, at step S557, the quality in the parameter in 
20 Fig. 56 is not satisfied, program control returns to step 
S552 for simulation of the period of time for the printing. 
As a result of the repetition of the above process, even 
when the quality is the one 3) designated by the user, 
the urgency period can not be satisfied. When, at step 
25 S556, the quality determination variable value is decre- 
mented by one, at step S557 the quality specified by the 
user is still not satisfied, and program control therefore 
moves to step S558. At step S558 a check is performed 
to determine whether the quality is the most important 
30 parameter When the quality is the most important pa- 
rameter, at step S560 the print job is changed in accord- 
ance with the specified quality, and the processing is 
normally terminated. 

For a case where the urgency is specified as the 
35 most important, when the quality is set to the minimum 
quality of 1, the printing period for the printer A521 is 50 
seconds, which conforms to the urgency setting. There- 
fore, the quality is set to the quality determination vari- 
able value (= 1), and the processing is normally termi- 
40 nated. 



[Nineteenth Embodiment] 



Fig. 57 is a diagram showing the transmission of a 
45 print job and other information in a system in this em- 
bodiment employing apparatuses. The thick arrows de- 
scribe the transmission routes for print jobs and the bro- 
ken line arrows describe the transmission routes for oth- 
er information. 

so The print job in this embodiment includes parame- 
ters that are referred to during printing: a document to 
be printed; the number of print copies; the print quality; 
and the print size. The other information includes sta- 
tuses, such as printing, normal, or out of paper of indi- 

55 vidual printers, and a schedule for a print job that is in- 
cluded in a print job scheduling unit 520. 

In Fig. 57, in a client machine 510, a print job gen- 
eration unit 512 generates a print job and stores it in a 
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print job memory unit 513. Then, the print job is trans- 
mitted by a print job transmission unit 514 in the client 
machine 510 to a server machine 511 for managing 
printers. 

In the server machine 511, a print job reception unit 
516 receives the print job and transmits it to a print job 
interpretation unit 571. The print job interpretation unit 
571 interprets an instruction from a user that is included 
in the received print job, and converts the instruction into 
information, such as a parameter, that can be processed 
by the system. When the parameters of the print job 
have not yet been set, the print job is transmitted to a 
print job automatic setup unit 517, and when the param- 
eters have already been set, the print job is transmitted 
to a print job automatic change unit 518. 

The print job automatic setup unit 517 sets print pa- 
rameters while taking into consideration the urgency, 
economy and quality values that are acquired from the 
print job. The print job automatic change unit 518 can 
perform the same process to change the parameters 
that have been set. The print jobs that are set or altered 
are registered in a print job schedule included in a print 
job scheduling unit 520 and are sequentially read to a 
print job transmission unit 51 9, or the print job is directly 
transmitted to the print job transmission unit 519 from 
the print job automatic setup unit 517 or the print job 
automatic change unit 518. The printing is then per- 
formed. 

The status, such as in printing, normal or out of pa- 
per, of an available printer is acquired by a printer state 
acknowledgement unit 515, and is transmitted to the 
print job automatic setup unit 517 and the print job au- 
tomatic change unit 5 1 8 where it is employed for the set- 
up or change of the print job. Similarly, the printing 
schedule for the print job included in the print job sched- 
uling unit 520 is also transmitted to these units 51 7 and 
518 for their employment. 

As is shown in Fig. 57, a print job may be transmitted 
along a different route. When a machine for generating 
a print job is the same as a machine for managing print- 
ers, the print job may not be transmitted via the print job 
transmission unit 514. 

In this embodiment, as is shown in Fig. 58 A, the pa- 
rameters of the print job are set ambiguously by using 
natural language. Fig. 58B is a diagram showing, rela- 
tive to an input character string in the natural language, 
the meanings represented by character strings accord- 
ing to the parameters and the items required to complete 
the meanings. The print job interpretation unit 571 refers 
to the meaning corresponding to an input character 
string in the natural language in Fig. 58B and the item 
required to complete the meaning, and interprets the in- 
struction given in the natural language that is received 
as a print job setup parameter as follows. 

From the character string "Print" in the input natural 
language, "printing" is the actbn that is the object of the 
user according to the meaning of the character string. 
Further, it is assumed that the required item "object" in- 



dicates the contents o1 a print job that was input at the 
same time, and that "quality" and "number of sheets" 
should be designated at a portion in the input natural 
language that has not yet been interpreted. Thus, the 

5 character string "one copy for client and five copies for 
members" is interpreted as an instruction to print one 
"sheet" having a high "quality" and five "sheets" having 
a desired "quality". 

The processes hereinafter performed by the print 

10 job automatic setup unit 517 and the print job automatic 
change unit 518 are the same as those in the seven- 
teenth and the eighteenth embodiments. 



[Twentieth Embodiment] 



IS 



In this embodiment, as is shown in Fig. 59, the print 
job parameters are set based on the distribution desti- 
nation described in a document, which is the contents 
of the print job, and on the number of sheets. 
20 The action of the job is printing. 

Printing for a "client" is interpreted as high quality 
printing, and the number of sheets is one. 

"ABC project" is interpreted as a "member use" and 
the printing quality is arbitrary, the number of sheets be- 
25 ing five. A user acquires in advance information that he 
or she is a member of the ABC project. 

The processes hereinafter performed by the print 
job automatic setup unit 517 and the print job automatic 
change unit 518 are the same as those in the seven- 
30 teenth and the eighteenth embodiments. 

[Twenty-first Embodiment] 

Fig. 60 is a diagram showing the transmission of a 

35 print job and other information in a system in this em- 
bodiment employing apparatuses. The heavy arrows 
describe the transmission routes for print jobs and the 
broken line arrows describe the transmission routes for 
another information. 

40 The print job in this embodiment includes parame- 
ters that are referred to during printing: the document to 
be printed; the number of print copies; print quality; and 
print size. The other information includes statuses, such 
as printing, normal, or out of paper, of individual printers, 

45 and a schedule for a print job that is included in a print 
job scheduling unit 520. 

In Fig. 60, in a client machine 510, a print job gen- 
eration unit 51 2 generates a print job, and stores it in a 
print job memory unit 513. Then, the print job is trans- 

50 mitted by a print job transmission unit 514 in the client 
machine 510 to a server machine 511 for managing 
printers. 

In the server machine 511 , a print job reception unit 
516 receives the print job and transmits it to a print job 
55 interpretation unit 571. The print job interpretation unit 
571 interprets an instruction from a user that is included 
in the received print job, and converts the instruction into 
information, such as a parameter, that can be processed 



21 



BNSDOCID: <EP 0853301 A2_l_> 



41 



EP 0 853 301 A2 



42 



by the system. When the parameters of the print job 
have not yet been set, the print job is transmitted to a 
print job automatic setup unit 517, and when the param- 
eters have already been set, the print job is transmitted 
to a print job automatic change unit 51 8. A print job sim- 
ulation unit 601 simulates printing at the setup that is 
specified by the print job to acquire the period of time 
for the printing and the consumption of toner. 

The print job automatic setup unit 517 sets print pa- 
rameters white taking into consideration the urgency, 
economy and quality values that are acquired from the 
print job. The print job automatic change unit 518 can 
perform the same process to change the parameters 
that have been set. The print jobs that are set or altered 
are registered in a print job schedule included in a print 
job scheduling unit 520 and are sequentially read to a 
print job transmission unit 51 9, or the print job is directly 
transmitted to the print job transmission unit 519 from 
the print job automatic setup unit 517 or the print job 
automatic change unit 518. The printing is then per- 
formed. 

The status, such as printing, normal or out of paper, 
of an available printer is acquired by a printer state ac- 
knowledgement unit 515. and is transmitted to the print 
job automatic setup unit 517 and the print job automatic 
change unit 518 where it is employed for the setup or 
change of the print job. Similarly, the printing schedule 
for the print job included in the print job scheduling unit 
520 is also transmitted to these units 517 and 518 for 
their employment. 

As is shown in Fig. 60, a print job may be transmitted 
along a different route. When a machine for generating 
a print job is the same as a machine for managing print- 
ers, the print job may not be transmitted via the print job 
transmission unit 514. 

Fig. 61 is a flowchart showing the processing per- 
formed by the print job simulation unit 601 First, at step 
S611, the print job simulation unit 601 initializes, in pre- 
paring for the simulation, a factor for determining the 
printing period and the consumption of toner. At step 
S612 the period of time for the printing is simulated at 
the setup that is also specified by the print job, and at 
step S613 the consumption of toner is simulated at the 
setup designated by the print job. In this manner, the 
period of time for the printing and the consumption of 
toner are acquired. 

A specific method is the same as that explained in 
the seventeenth and the eighteenth embodiments. 

[Twenty-second Embodiment] 



In this embodiment, a plurality of output trays are 
provided for a printer, and a tray is selected in conso- 
nance with a user. 

Fig. 62 is a flowchart showing the processing for a 
printer system in this embodiment. When a printer sys- 
tem receives a print job. first it performs a process for 
coping with a case where a plurality of print jobs are re- 



ceived at the same time, and then it initiates the process- 
ing shown in Fig. 62. 

Figs. 63A and 63B are diagrams showing the exter- 
nal appearances of the printer systems in this embodi- 
5 ment. As is shown, printers 631 and 632 each have a 
plurality of output trays: for the printer 631 , each output 
tray is provided with a screen for displaying the name of 
the person to whom a document on the tray belongs; 
and for the printer 631 . the names of the persons whose 
10 document remain on which trays are displayed on a 
common screen. When a user is set in advance for each 
output tray, the individual trays are user dedicated trays. 
When the trays are not specifically assigned, the users 
of the output trays are changed as needed. 
75 The printer system in this embodiment implements 
the functions of notifying a user of the receipt of a doc- 
ument, the termination of printing, and the failure to col- 
lect a printed document (uses a sensor). 

The above processing will now be described. In Fig. 
20 62, at step S621 information concerning a destination is 
acquired from information included in a print job. The 
information concerning the destination is obtained, as is 
explained in the twenty-ninth embodiment, either by in- 
terpreting the setup described in the print job, or by ex- 
25 tracting it from information, such as E-mail, that is set 
separately from the contents of the print job. At this time, 
when, as in conventional use, a printer is employed as 
an output device for a personal computer, usually an ad- 
dressee is not designated. When the printer is employed 
30 as a facsimile machine or for transmission of E-mail, as 
in the system in this embodiment, an addressee is nor- 
mally designated. 

At step S622 a check is performed to determine 
whether or not an addressee is designated. At step 
35 S623 the addressee is notified of the arrival of a docu- 
ment. At step S624 the addresser information is ac- 
quired from the information included in the print job. The 
addresser information is acquired, as is explained in the 
twenty-ninth embodiment, either by interpreting the des- 
40 ignation described in the print job, or by extracting it from 
information, such as E-mail, that is set separately from 
the print job. When, as in conventional use, a printer is 
employed as an output device for a common personal 
computer, normally the addresser falls within the control 
45 sphere of the system. However, when the printer is used 
as a facsimile machine, as in the system in this embod- 
iment, usually the addresser is not within the control 
sphere. At step S625 a check is performed to determine 
whether the addresser is in the control sphere. At step 
50 S626 when the addressee is designated, a correspond- 
ing output tray is selected. When the addressee is not 
specified and only the addresser is specified, an output 
tray corresponding to the addresser is selected. At step 
S627 the addresser or the addressee information is 
55 transmitted to the corresponding output tray, as is 
shown in Fig. 63A. Or, as is shown in Fig. 63B. the ad- 
dresser or the addressee inf ornnation and the output tray 
that is employed are displayed on the common display. 
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When the user is set for each output tray, the individual 
trays are user dedicated trays. When the trays are not 
specifically assigned, the users of the output trays are 
changed as needed. 

At step S628 printing is executed. At step S629 a 
check is performed to determine whether the printing 
has been terminated and whether the addresser is with- 
in the control sphere. At step 5630 the addresser is no- 
tified of the termination of the printing. At step S631 a 
check is performed to determine whether a document 
has been left for a predetermined period of time and 
whether the addresser is in the control sphere. At step 
S632 the addresser is notified that a printed document 
has been left in the tray for an extended time period. 

[Twenty-third Embodiment] 

In this embodiment, information concerning the 
time is handled. Fig. 65 is a diagram showing the func- 
tional arrangement of a system according to the embod- 
iment. A processor 650 includes an understanding unit 
652, a planning unit 653, an execution unit 654, a re- 
sponse unit 655 and a knowledge base 656, and is con- 
nected to a database 651 . 

Fig. 64 is a flowchart showing the processing per- 
formed by the functional arrangement in Fig. 65 in this 
embodiment. 

At step S640 a check is performed to determine 
whether or not there is external input. If so, at step S641 
the understanding unit 652 employs the knowledge 
base 656 to analyze an input document as the units of 
words and sentences, and at step S643 the object of the 
input contents is analyzed and ascertained. At step 

5644 a check is performed to determine whether infor- 
mation required for an understanding of the object is in- 
sufficient. If the information is unsatisfactory, at step 

5645 a query is issued to a user, and a response from 
the user is studied in the knowledge base 656. Program 
control thereafter returns to step S643. If the information 
is adequate, program control moves to step S646. 
When, at step S640, there is no input, at step S642 a 
job to be executed is found and program control advanc- 
es to step S646. 

At step S646 the planning unit 653 prepares a plan 
to achieve the object. At step S647 a check is performed 
to determine whether information required for planning 
is insufficient. If the information is unsatisfactory, at step 
S648 a query is issued to the user, and a response from 
the user is studied in the knowledge base 656. Program 
control thereafter returns to step S646. If the information 
is adequate, at step S649 the execution unit 654 exe- 
cutes the plan. At this time, the execution unit 654 ac- 
cesses the database 651 or communicates with another 
application, as needed. At step S650 the response unit 
655 determines whether or not a response should be 
made. If a response should be made, a decision is made 
concerning the selection of the contents for the re- 
sponse. Further, at step S651 a response is prepared 



in accordance with the selected contents, and is trans- 
mitted to the user. 

The understanding unit 652 acknowledges, as 
needed, externally input information, such as keyboard 

5 input, voice input. E-mail or facsimile or news informa- 
tion, analyzes the received information and also the time 
concept contained in the words in sentences, and ob- 
tains an understanding of the contents of the sentences 
that are concerned with time. 

10 Fig. 67 is a diagram showing an example where 
schedule information is extracted from electronic mail. 
In the above process, specifically, as is shown in Fig. 
67. the contents of a document received by E-mail are 
analyzed, and the time concepts such as "February 

15 22nd^ "13:00" and "15:00" are found and analyzed. 
Then, an understanding of the sentence concerning the 
concept, "We have scheduled a patent system explana- 
tion meeting at conference room Bfrom 13:00 to 15:00 
on February 22" is obtained, and an action associated 

20 with the time can be correlated with the schedule. 

The analyzed time concept is compared with the 
current time. If the time concept describes the future, a 
corresponding action is registered in the schedule, but 
if the time concept describes the past, it is abandoned 

25 without being registered or it is registered as an infor- 
mation of a type other than that for a schedule, such as 
information for a personal data base, that is used to 
record past events. 

The understanding unit 652 analyses the contents 

30 of externally input information, such as keyboard input, 
voice input. E-mail, facsimile or news information. When 
the understanding unit 652 finds in the sentences a term 
(a word or a concept) that is unknown to the system, or 
a plan that the system has not yet executed, and when 

35 the word or the plan that is detected is not urgent, the 
understanding unit 652 studies the term or the plan by 
sequentially searching for information concerning it, and 
stores it as knowledge in the knowledge base 656. 
Specifically, when, for example, the unknown term 

40 XYZ appears, the presence of XYZ is memorized, and 
sentences "XYZ is very delicious" and "XYZ is hard" are 
analyzed to extract the concept that XYZ is hard, deli- 
cious food. 

Similarly, when a new plan is instructed and when 
45 from the current situation It is understood to be a plan 
for use when a user is absent, the understanding unit 
652 studies the plan as an example of how to make a 
plan for when a user is absent. 

In addition, when, for example, a system has been 
50 acquiring information concerning a user from news and 
notifying the user each time such information is found, 
and then upon the receipt of a notification that the user 
responds by sending the system an instruction in natural 
language that "from now on, there is no need to take 
55 notice of this information" the system can ascertain that 
transmission of the information is no longer necessary. 

As is described above, in the system, the under- 
standing unit 652 analyzes the contents of externally in- 



23 



BNSDOCID: <EP 0e53301A2J_> 



45 



EP 0 853 301 A2 



46 



put information and also analyzes the object. The plan- 
ning unit 653 prepares a plan of an action for the system 
to take to achieve the object. The execution unit 654 per- 
forms the processing. And the response unit 655 notifies 
a user of the results of the processing. 

When, for example, it is ascertained, from informa- 
tion concerning an addresser, upon the receipt of the E- 
mail message shown in Fig. 67 that the mail is from an 
addresser who is not registered in the database, infor- 
mation concerning the addresser is extracted from the 
message and is registered in the personal database in 
the database 651 . Further, when more information is ob- 
tained concerning the addresser who is not registered 
in the personal database, the information may be regis- 
tered in the personal database. 

Since the understanding unit 652 understands the 
message is for a notice for an explanation meeting, it 
accesses the schedule of a user in the database 651, 
and compares new schedule data with the current 
schedule data. When the new schedule data and the 
current schedule data do not conflict, the new data are 
additionally registered in the schedule. However, as is 
shown in Fig. 69, when the schedule data conflict, a re- 
ply to the addresser and a notice for a user are prepared. 

As is described above, since this system under- 
stands the contents of received information, the system 
can prepare a response in consonance with the situation 
and execute it. 

In the above explanation, the system prepares a re- 
ply to a user who sent the E-mail without confirming it 
with a user, and transmits the reply. This is because the 
sentence, "Please contact me as soon as possible if 
there is any difficulty" is in the message, and as a result 
of the analysis of this sentence, it is determined that a 
response is very urgently required. If the urgency of a 
response is determined to be low because a date for the 
holding the meeting is fairly advanced and the attend- 
ance at the meeting is arbitrary, the system confirms it 
with a user before transmitting a response. In other 
words, the system determines in which action to take in 
consonance with the degree of urgency. 

In addition, the schedule or the personal database 
can be accessed to transmit a response, relative to the 
input of the natural language by a user, as well as a nat- 
ural language instruction received fronn a user 

For example, the personal database can be ac- 
cessed by asking the natural language question, "What 
is the telephone number of Mr XX?". Or the schedule 
can be accessed when the natural language question, 
"Where will the meeting on the 16th be held?" is re- 
ceived, to send a reply to a user 

Fig. 66 is a diagram showing the Input/output types 
between the system that performs the overall process- 
ing in this embodiment and external devices. 

Input data can be data input via a keyboard, natural 
language information received by E-mail, documents or 
pictures input as images by a scanner, voice input via a 
microphone or images input by a camera. A character 



recognition process is preformed for a document that is 
read by the scanner, or a voice recognition process is 
performed for speech, so that input information in natu- 
ral language can be obtained. 

5 As for when there is no input (idle state), as is shown 
at step S642 in Fig. 64, when there is no external input 
the system searches for a job that it should execute and 
regards it as input. In the idle state, when no job is given 
to the system, the system, for example, accesses news 

10 and aggressively acquires as input information concern- 
ing the user 

As for output, there are the registration of a sched- 
ule in a database, the filing or erasure of data relative 
to a file memory device, or the dispatch of a notice to a 

15 user or a reply to an addresser The importance of the 
output is that it can be determined that a process can 
not be performed within the control range of a system, 
and another action, such as transmitting a response to 
that effect, can be performed. 

20 An input/output destination is a user or an external 
device, or a different processor in the system or another 
application. 

In addition, input documents are analyzed, and data 
concerning a person and data concerning the time are 
2S extracted and stored in the personal database and the 
schedule database. Also, data required for the anticipa- 
tion of an action is extracted to use for planning. 

In the example in Fig. 67, as a result of analysis of 
an E-mail document that is input, the following descrip- 
30 Won concerning a person is acquired: 
To: toshima@abc.canon.co.jp, 
rohra@abc, canon. co.jp, 
kazuyo @ abc.canon.co.jp 
From ichiro@abc.canon-CO.jp 
35 I am Suzuki of system promotion section. 
Suzuki (ichiro@abc.canon-co.jp) 
044-123-4569 (ext. 654-3210) 

Canon Inc. Intellectual Property Head Office System 
Promotion Section 

40 Assuming that a person whose mail address is "ka- 
zuyo© abc.canon.co.jp" is already registered in the per- 
sonal database and the other persons are not regis- 
tered, as is shown in Fig. 67, data for PERSON 1 , 2 and 
3 are extracted and registered in the database 651 . 

45 Since there is a description concerning the time, 
"We have scheduled a patent system explanation meet- 
ing at conference room B from 13:00 to 15:00 on Feb- 
ruary 22" data for EVENT1 is extracted. 

Further, since there is a description used for esti- 

50 mating an expected action, "Please contact me as soon 
as possible if there is a difficulty" this is used for the 
processing to be explained while referring to Fig. 70. 

Fig. 68 is a specific flowchart for step S649 pre- 
pared by extracting the portions from the flowchart in 

55 Fig. 64 that are required for performing the processes 
in Figs. 67 and 69. 

The processing will how be explained while refer- 
ring to Figs. 67 and 69. 
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At step S680 E-mail is input and at step S681 . of 
each word and sentence, Ihe input E-mail document is 
analyzed. At step S683 it is ascertained from a signature 
and a header that "there is information concerning per- 
sons" and "an event, an explanation meeting, will be 
held". Further, it is ascertained that "a reply is requested 
if there is a difficultyV 

At step S684 the data concerning a person is ex- 
tracted and a plan for registering the data in a database 
and a plan for registering the explanation meeting event 
in the schedule are made. At step S685, it is ascertained 
that there is a plan to be executed. At step S686 It is 
found that there is no problem with the plan, and pro- 
gram control thereafter moves to step S688. At step 
S68e the data concerning the person is extracted and 
registered in the database, and program control returns 
to step S685. 

In Fig. 67, at step S685 there is a plan to be exe- 
cuted: registration of the explanation meeting event in 
the schedule. At step S686 it is assumed that the event 
can be registered in the schedule with no problem, and 
program control goes to step S688. At step S688 the 
event is registered in the schedule, and program control 
returns to step S685. At step S685 there is no other plan 
to be executed, and at step S689 no acknowledgement 
receipt is required. The processing is thereafter termi- 
nated. 

In Fig. 69, at step S685 there is a plan to be exe- 
cuted: registration of the explanation meeting event in 
the schedule, and at step S686 a conflict is found in the 
schedule. Since there is a problem with the execution 
of a plan, at step S687 a reply to that effect is transmitted 
and a plan is made to notify a user that there is a prob- 
lem. At step S585 there is a plan for writing a reply and 
transmitting it, and at step S686 no problem concerning 
the execution of the plan is found. At step S688. there- 
fore, a reply indicating that the timing is inconvenient is 
prepared and transmitted to the addresser At step S685 
there is a plan for preparing a document for notification 
of a user At step S686, there is no problem in execution, 
and at step S688 a response to a user Is prepared de- 
scribing that a reply was sent for reconsideration be- 
cause of the contents of the E-mail and the conflict of 
the schedules. At step S685 there is no plan to be exe- 
cuted and at step S689 since there is an notice to a user, 
it is transmitted to the user. The processing is thereafter 
terminated. 

In the example in Fig. 69, when the schedule infor- 
mation EVENT1 that is extracted is compared with the 
schedule EVENT2 that is planned previously, it is found 
that the two events conflict Further, according to the in- 
formation extracted in the example in Fig. 69, it is ap- 
parent that a reply is necessary if there is a difficutty. In 
the system in this embodiment, the knowledge for pre- 
paring letter is employed to write a letter describing that 
a user can not attend the meeting because the sched- 
ules are conflicting, and the reply is automatically trans- 
mitted. 



In addition, the user of the system in this embodi- 
ment is notified that the system automatically has trans- 
mitted a reply for the user 

5 [Twenty-fourth Embodiment] 

Fig. 70 is a diagram showing an example where a 
user sends a question by voice to a system according 
to this embodiment. 
10 When a user asks by voice "where will tomorrow's 
conference be held?", the system can examine the 
schedule information of the user and tell the location of 
the conference. 

When the user does not know how to get the loca- 
is tion for the conference and asks "In which area in Shi- 
momaruko?". the system examines the database, per- 
forms planning for an explanation to make a user under- 
stand, prepares a briefing story, and uses graphical im- 
ages and sounds to explain how to get the location. 
20 Fig. 71 is a flowchart showing the processing per- 

formed in Fig. 70 by referring to the basic flowchart in 
Fig. 64. An external input is regarded as a query. 

The processing relative to a first inquiry will be ex- 
plained. At step S71 0 voice (where will tomorrow's con- 
2S ference be held?) is externally input, and at step S711 
the input sentence is analyzed. At step S712 it is under- 
stood that the input sentence is an inquiry for the con- 
ference, and that the object of the user is to know the 
location of the conference. At step S713 the following 
30 planning is performed to answer the inquiry. (1) Tomor- 
row's schedule is extracted. (2) The location registered 
in the schedule is acquired. (3) A reply is prepared to 
tell the acquired location. 

At step S714 information is sufficient and program . 
35 control moves to step S7 16. If the schedule show a plu- 
rality of conferences, the system does not identify which 
conference is. Therefore, at step S715 a query is sent 
to a user to remake a plan. At step S716 the plan made 
at step S713 is executed, and a reply document to the 
40 user is prepared. At step S71 7 it is ascertained that there 
is a response to the user, the system determines the 
transmission of the response. At this time, since it is un- 
derstood that the input was done by voice, transmission 
of the reply by voice is also determined. At step S718 
45 the response, which is voice data obtained by conver- 
sion, is transmitted to the user. 

The processing is temporarily terminated, and fol- 
lowing this, an inquiry from the user is input. The process 
relative to the second inquiry will now be described. 
so At step S710, voice (In which area in Shimomaru- 
ko?) is input externally, and at step S711 the input sen- 
tence is analyzed. At step S71 2 it is understood that this 
inquiry is a continuous question of the first one, and also 
that a specific location of the place name that was an- 
55 swered is being asked. 

At step S713 the following plans are made to re- 
spond the question: (1) a plan for explaining that it is 
difficult to explain the location by writing; (2) a plan for 
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explaining the location by sequentially sending images; 
(3) a plan for acquiring necessary images from an image 
database and a location database; and (4) a plan for 
preparing sentences corresponding to Images. 

At step S714 the information is adequate, and at 
step S71 6 the plans made at step S71 3 are sequentially 
executed and a reply to a user consonant with the im- 
ages is prepared. At step S717, since there is a reply 
document to be transmitted to the user, the transmission 
of the response is determined. At this time, it is under- 
stood that the input was done by voice, so that it is also 
determined that explanation will be given by voice while 
displaying images on a monitor. At step S718 the re- 
sponse is transmitted to the user. The processing se- 
quence in Fig. 70 is thereafter terminated. 

[Twenty-fifth Embodiment] 

In an example shown in Fig. 72, when two sched- 
uled events conflict, the order of priorities of the two 
events is obtained from a comparison, and a plan to han- 
dle this problem is prepared and proposed to a user. 

That is, in this example, when one schedule is found 
to be more important than the other, it is proposed that 
the less important schedule be canceled. 

If the user knows somebody to send as his or her 
proxy to the scheduled event for which cancellation is 
proposed, the user can send the system a response to 
that effect. The system can then perform an action in 
consonance with the response. Also, the system can 
study a plan for despatching another person as a proxy 
In Fig. 72 is shown a combination of other methods 
performed at step S646 in Fig. 64 to make plans when 
the schedule conflict is found at step S686 in the flow- 
chart in Fig. 68. 

The processing in Fig. 68 has been explained for 
unconditionally transmitting to a user a response for an 
inconvenience. In this embodiment, the processing for 
a plurality of plans is employed as the re-planning meth- 
od at step S646. 

Fig. 73 is a flowchart for evaluating a priority extract- 
ed from the contents of a document and for proposing 
it to a user. The processing will now be explained. 

The priorities of the scheduled events is determined 
by using information that the user has set in advance for 
each schedule, attendance at an event included in the 
schedule, the object of a scheduled event and the result 
obtained as a result of the analysis of the received doc- 
ument. A user may set the priorities of persons that are 
expected to attend the pertinent event. 

The priority determined according to the object of 
the schedule may be set in advance by a user, or may 
be determined while taking into account the object or 
the field that the system understands, from the study of 
the past, that the user is interested in. That is, when the 
user engages in computer associated work, the sched- 
ule for a computer associated event priority over an 
event associated with real estate. Of course, if a user 



plans to purchase a house and that data is stored as 
information concerning the user, the schedule of an 
event associated with real estate may take priority 
For the priority determined from the result of the 

5 analysis of the document, when a document including 
the sentence "Please be sure to attend the meeting" and 
a document including the sentence "Please attend the 
meeting if possible" setup a conflict, the first document 
received is determined to take priority. 

10 In Fig. 73, at step S730 the current schedule infor- 
mation is acquired. At step S731 the priority for new 
schedule information is evaluated. At step S732 the pri- 
orities of both schedules are compared. At step S733 
from the result of the comparison, a plan Is made to pre- 

15 pare a notice for querying a user (e.g., display two con- 
flicting scheduled events and ask whether the event 
having the lower priority can be canceled). At step S734 
the prepared notice is transmitted to the user. At step 
S735 a response from the user relative to the inquiry 

20 concerning the plan is acquired. The same re-planning 
process as in Fig. 68 is performed to again prepare a 
plan by using the response obtained from the user so 
that an action consonant with the response of the user 
can be taken. 

25 Fig. 74 is a flowchart showing the re-planning proc- 
ess. 

At step S740 the response finally obtained in Fig. 
73 is regarded as input. At step S741 the input sentence 
is analyzed. At step S742 it is ascertained that the 

30 schedule is to be changed and that this Is the first in- 
structed plan. 

At step S743 the following plans are made: (a) the 
changing of the schedule; (b) the preparation of a doc- 
ument to transmit the schedule to a proxy; (c) the study 

35 of anewplan; (d)the preparation of a document to trans- 
mit the new plan to the user; and (e) the notification sent 
to a user concerning another schedule that should be 
recalled. 

At step S744 the plans made at step S743 are ex- 
40 ecuted: (a) the schedule is changed; (b) a document is 
prepared to transmit the schedule to a proxy; (c) a new 
plan is studied; (d) a document Is prepared to transmit 
the new plan to the user; and (e) a user is notified of 
another schedule that should be recalled. 
45 At Step S745 the notification document is transmit- 
ted to the user. The processing for performing the proc- 
ess in Fig 72 is thereafter terminated. 



[Twenty-sixth Embodiment] 



so 



Fig. 75 is a diagram Illustrating the overall image of 
a system according to this embodiment, Including input/ 
output devices. Fig. 76 is a flowchart showing the 
processing performed by the system In Fig. 75. 
55 Information Input by E-mail, by voice, via a key- 
board; across the WWW, and by input devices, such as 
a telephone, a facsimile nr^chine, a scanner and a cam- 
era, is analyzed by an input management unit 751 (step 
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5760) . A core unit 752 ascertains the contents of the 
information and plans an appropriate process (step 

5761) . An output management unit 753 determines an 
output medium and prepares the contents to be output 
and an output route (step S763). Then, the information 
is output by E-mail, by voice, across the WWW, or by 
output devices, such as a telephone, a facsimile ma- 
chine, a printer and a copier. 

Figs. 77 to 79 are specific flowcharts showing the 
process at step S760 (the process performed by the in- 
put management unit 751), at step S761 (the process 
performed by the core unit 752) and at step S763 (the 
process performed by the output management unit 
753). 

In Fig. 77, when, at step S770, it is determined that 
there is new input, at step S771 the input information is 
obtained. At step S772 the input information is analyzed, 
as needed, after the information has been identified. At 
this time, the input management unit 751 accepts infor- 
mation from various media and identifies or analyzes the 
data using methods corresponding to the individual me- 
dia. 

in Fig. 78, the core unit 752 receives the information 
that is analyzed in Fig. 77. 

At step S780 the object of the input information is 
analyzed and ascertained from the result of the analysis. 
At step S781 how the object of the input information is 
related to the main apparatus (the system or the user) 
is examined- At step S782, from the relationship with the 
main apparatus, planning for what to execute is per- 
formed in consonance with the object. At step S783 the 
plan made at step S782 is executed. 

In Fig. 79 the output management unit 753 employs 
the results obtained in Fig. 78 to determine whether a 
response is needed. If a response is required, it Is pre- 
pared and output 

At step S790 the result obtained in Fig. 78 is ana- 
lyzed. At step S791 a check is performed to determine 
whether or not there Is a response relative to the results. 
When there is no response, the processing Is thereafter 
terminated. If there Is a response, at step S792 a re- 
sponse relative to the external response Is determined. 
At step S793 a medium for the response is determined. 
At step S794 a response is prepared in consonance with 
the designated medium. At step S795 the prepared re- 
sponse is issued by the designated medium. At step 
S896 the response is actually output. The processing 
performed by the arrangement in Fig. 75 has been ex- 
plained. 

When, for example, E-mail is received that notifies 
the user of the holding of a conference, the input man- 
agement unit 751 analyses the input information accord- 
ing to the procedures In Fig. 77. From the obtained re- 
sult, the core unit 752 prepares a plan, according to the 
procedures in Fig. 78, as to how the user and the system 
should handle the information. The plan is transmitted 
to the output management unit 753, which in turn per- 
forms the process according to the procedures in Fig. 



79. When a response should bie transmitted with the 
plan, a response is actually prepared and transmitted. 

Depending on the contents of the conference de- 
scribed in E-mail message and the status of the sched- 

5 ule of the user, at step S782 a plan is prepared for the 
aggressive transmission of the response to the user, so 
that preparation of the response is required. At step 
S791, therefore, it is determined that a response is to 
be transmitted. At step S792 the contents of the re- 

10 sponse are determined. When the schedules conflict 
and when it is uncertain which schedule should be se- 
lected or whether a user intends to attend a specific con- 
ference, the contents of a response asking for a decision 
are determined. At step S793 a medium is selected by 

IS which the contents of the response can be transmitted 
most effectively. When, for example, the user is outside 
the office, a medium, such as a telephone or a facsimile 
machine, is selected that can issue a notice to the user. 
When the user is being operating a personal computer, 

20 a medium, such as a personal computer, is selected that 
can effectively display various reference materials. At 
steps S794 and S795 a document and an image are pre- 
pared in consonance with the contents and the medium 
that are decided above, and a response is prepared us- 

2S ing a method for converting the data into voiced natural 
language in consonance with a specific medium, and is 
issued. 

Fig. 80 is a detailed flowchart for Fig. 75 to explain 
the processing for extracting data from input informa- 
30 tion. In this system according to the embodiment, when 
a document is input, at step S801 type identification 
symbols, such as the layout and a bar code of a docu- 
ment, are employed to infer the document type. If the 
document is inferred to be a letter, a report or a patent 

3S publication, program control moves to step S803. When 
no identificatbn of the document is possible, program 
control goes to step S810 whereat OCR is thoroughly 
performed to ascertain the document type. 

At step S803, in order to confirm the inferred docu- 

40 ment type is correct, a characteristic block is interpreted 
by the OCR to identify the document type (see Figs. 82A 
and 82B). At step S804 the addressee of a letter and 
the patent serial number of the patent publication are 
found and the document type is confirmed. When the 

45 confirmed document type matches the inferred docu- 
ment type, program control advances to step S806. If 
the document types do not match, program control 
moves to step S810. For slips, at the top of which char- 
acter strings describing the slip type are printed, the doc- 

so ument type can be easily determined by the OCR on the 
employing upper portion of the sheet. 

At step S806 the knowledge base of the document 
type that has been confirmed is employed to read and 
interpret a specific block using OCR. At step S807 a 

55 check is performed to determine whether there was in- 
formation in the past that concerns the resultant infor- 
matbn. If such information was present, program con- 
trol moves to step S808. If such information was not 
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present, program control goes to step S810. As a result, 
the addressee or the patent number is found and wheth- 
er the information is a reply to a letter that was sent in 
the past can be ascertained. 

At step S808, based on the identified status, the ob- 
ject is decided from the contents of the important text, 
and at step S809 a process to be executed is deter- 
mined. 

At step S811, because of the above object, the in- 
formation is analyzed in another range where OCR has 
not yet been performed. At step S812 a required proc- 
ess is actually performed. As a result, a filing process, 
etc., is performed as needed. 

In the above process, the OCR can be performed 
while inferring the contents of the information, more ef- 
ficient and precise OCR results can be obtained than 
when, as m the conventiooril case the OCR use is ef- 
fected from the beginning Accordingly, the process 
based on the OCR rosutts can bo performed precisely. 

Fig. 81 is a dugfrtm showing an example letter/fac- 
simile that IS a ta:gcl to* data exiiaction 

In this example ai step SoOl the document type is 
inferred from the layout of a document As a result, Doc- 
Type! = letter/fax. 

At Step S803 a Gpoctltc block is read by the OCR 
and is interpreted (soo Figs 82A and 82B). Then, 
"From", "To** and "Dear Str" are acquired, and at step 
S804 DocType2 - letter/fax. 

The process for reading a specific block by using 
OCR. which was explained at step S803 in Fig. 80. will 
be specifically described. Figs. 32A and 82B are dia- 
grams for explaining this process. 

The system in this embodiment scans a specific 
block as follows: 

(1 ) A specific block is read fast by pre-scanning with 
a low resolution. 

(2) A form of the information that Is read is com- 
pared with an information form stored in a DB 823, 
When the forms match, input information other than 
the form, or information in an area (e.g., inside the 
frame of a card in Figs. 82 A and 82 B) specified for 
each form, is read at a high resolution. 

(3) Only the area that is read is regarded as an OCR 
target and analyzed, and the processing is contin- 
ued. 

In addition, since in this example it is assumed that 
there are a plurality of cards having the same form, doc- 
uments that are to be read at a predetermined interval 
probably have the same form. Therefore, first, the proc- 
ess is performed while it is assumed that the documents 
have the same form, and when a contradiction occurs, 
it is assumed that the documents have different forms 
and a re-analysis of them is performed. 

As a result, the processing speed is considerably 
increased, the analysis range Is limited, and an analysis 
domain is determined by specifying a form, and the anal- 



ysts precision is drastically improved. 

The determination of the object from the contents 
of the information, which was explained at step S808 in 
Fig. 80, will be specifically described. 
s The system in this embodiment employs date data 

to perform the determination process according to the 
procedures in Figs. 83A and 83B. When a sender is a 
user, the following process Is performed. 

10 (1 ) When the current date Is today, transmission by 
facsimile is decided. 

(2) When the date is yesterday or some time in the 
past, the user is queried as to whether the same 
document was received before or whether he or she 

^5 has seen the same contents, in order to determine 
whether to perform filing, re-transmission, or wheth- 
er the user made a mistake. 

(3) When the date is tomorrow or some day in the 
future, the user is queried in order to determine 
whether the information should be held until the 
designated date, or whether the user made a mis- 
take. 

More specifically, assuming that in a document in 
25 Fig. 81 , Datel is the date of a document, Date2 Is the 
today's date, 

( 1 ) Date 1 ~ November 25, 1 996 & Date2 = Novem- 
ber 25, 1 996 ~ send the document 
30 (2) Datel ^ November 25, 1996 & Date2 ^ July 2, 

1 996 - - filed 

(3) Datel ^ November 25, 1996 & Date2 = Novem- 
ber 26, 1996 

35 Date of contents = December 18 and 22, 1996 - 
mistake 

Figs. 83A and 83B are flowcharts showing the 
processing for analyzing the object extracted from an 
input document. When a document is Input, at step S830 

^0 a check is performed to determine whether the sender 
Is a user. If the sender is a person other than a user, the 
input document is determined to be a received docu- 
ment. At step S831 a corresponding object, such as fil- 
ing or data extraction, is examined, and the processing 

45 is thereafter terminated. 

When the sender is a user, program control advanc- 
es to step S832 to analyze the type of the input docu- 
ment. At step S833 the date of the document is com- 
pared with the current date. When the two dates are 

50 close to each other, at step S835 a check is performed 
to determine whether the document was transmitted be- 
fore. When the document was not transmitted before, 
its object is determined to be "transmission of a docu- 
ment". If the same document was transmitted before, at 

55 step S843 the object for filing or re-transmission Is ex- 
tracted. 

When the date of the document Is a fairly old date, 
program control moves to step S836, whereat a check 



28 



BNSDOCID: <EP ^0653301 A2J_> 



55 



EP 0 853 301 A2 



56 



is performed to determine whether or not the same doc- 
ument was transmitted before. When the document was 
transmitted before, program control moves to step 
S843. When the document was not transmitted, pro- 
gram control goes to step S837 whereat a check is per- 
formed to determine whether any other date is de- 
scribed in the information for the document. When there 
is another date, program control goes to step S843. 
When no other date is found, it is ascertained that the 
object can be transmission of the document, and also 
that the date may be wrong. 

When the date of the document is a date fairly far 
in ad'-rance at step S840 a check is performed to deter- 
mine whether any other date is described in the infor- 
mation lor the document. When, at step S841 , there is 
another dale the object is determined to be "transmis- 
sion ot the document". When no other date is found, pro- 
gram control fTKDvos to step S839. 

ITwonly-sevenUi Embodiment] 

Fig 64 IS rt diagram showing an example arrange- 
ment ol a system according to this embodiment. In this 
embodiment the system serves as a current facsimile 
machine 

According to the system in this embodiment, with- 
out a user spocilying an addressee, the addressee for 
a document is determined from bar code on a document 
that IS read or information on a cover sheet, so that the 
document can bo transmitted to a correct addressee. 

As a result, appropriate information can be trans- 
mitted to a printer or a facsimile machine, by E-mail or 
via the WWW 

(Twenty-eighth Embodiment] 

Fig. 85 is a diagram illustrating an example arrange- 
ment of a system according to this embodiment. In this 
example, the system implements a desk sorter function 
for processing ail the kinds of documents on the desk. 

That is. in consonance with the contents of the doc- 
ument that is read, sorting, filing, scheduling, data ex- 
traction and automatic processing are performed. 

[Twenty-ninth Embodiment] 

Fig. 86 is a detailed diagram for explaining the sys- 
tem in Fig. 85. The processing performed by the system 
shown in Fig. 86 will be specifically described while re- 
ferring to the flowchart in Fig. 87. 

Fig. 87 is a flowchart showing the processing for this 
embodiment. Figs. 88 and 89 are tables for knowledge 
used during the processing; the table in Fig. 8S shows 
the knowledge in general knowledge base, and the table 
in Fig. 89 shows the knowledge in a knowledge base of 
a field specified on a cover sheet. In this embodiment, 
information defined in the tables in Figs. 88 and 89 is 
employed to compare a character string included in an 



input document with a character string defined in a char- 
acter string column in the table, and the conception in a 
corresponding column, a Role for further specifying the 
meaning, and a Condition for specifying succeeding in- 
5 formation, or an instructed process Action are acquired, 
and analysis for them is performed. 

In Fig. 87, at step S870 a cover page is scanned 
and an OCR process is performed. At step S871 the 
knowledge base in Fig. 88 is examined to extract from 
10 it data, such as the names of a sender and a receiver 
and their telephone numbers and facsimile numbers. At 
step S872 the obtained data are registered in a data- 
base. In the example in Fig. 86, character string "To: 
Macrohard Corp." is included in the cover page of a re- 
is ceived document. When this character string is com- 
pared with an item defined in the character string column 
in Fig. 88. Then, matching item "To" is found and its Role 
is a reception company name or an individual name, so 
that "Macrohard Corp." is extracted as information con- 
20 cerning a receiver. 

At step S873 a notification method and means, a 
filing job, an action for a place, and a job are extracted. 
In the example in Fig. 86, character string "File: MH/ 
Contract" is included in the cover page of the received 
2S document. When this character string is compared with 
the items defined in the character string column in Fig. 
89, matching item "File" is found, filing is extracted from 
the column as an instructed action, and ■MH/Contract" 
is extracted from the document as information for a filing 
30 place. 

Assuming that Datel November 25. 1996 and 
Date2 = November 25, 1996, as is explained in Figs. 
83A and 83B, it is determined that the object is trans- 
mission of a document. ^ 

35 At step S874 a check is performed to determine 
whether an action for the object of the processing is 
present. Since the transmission of the document to a 
receiver is the object, at step S875 a transmission action 
is performed. At step S876 the information on the cover 

40 page is employed to determine whether the filing is nec- 
essary. Since the filing is required in the example in Fig. 
86, at step S877 the document is filed at the instructed 
place, "MH/Contract". 

At step S878 as well as at step S876 the information 

45 on the cover page is employed to determine whether or 
not a notification for the transmission of the document 
should be issued to the sender. Since it is instructed to 
notify the sender, by voice, of the transmission of the 
document, at step S879 the sender is notified by voice 

50 of that the document has been transmitted to the receiv- 
er At step S880 whether or not another action is deter- 
mined. At step S881 a keyword is designated, and an 
index is prepared by using keyword ABC. 

55 [Thirtieth Embodiment] 

Fig. 90 is a diagram showing an example process- 
ing where the history in the past is referred to based on 
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an instruction by voice in natural language. The 
processing in Fig 90 will be explained while referring to 
the flowchart in Fig. 91. 

Fig. 91 is a flowchart showing the processingforthis 
embodiment. When an aural instruction "Fax Contract s 
again to John. Notify htm by Phone" is received from a 
user, the system analyzes the object of the instruction 
in the natural language, and understands "Re-transmit 
document "Contract" to John, and Notify him of the 
transmission". Thus, the processing in Fig. 91 is initiated 
to specify the document Contract and the addressee 
John. 

Since re-transmission means that the document 
was previously transmitted, at step S910 history infor- 
mation is acquired from the database. Then, the docu- 
ment Contract and the addressee John are specified 
from the history information. At step S911 the address 
of the John is obtained from the personal database. 
Since, in the database, there are two Johns, John Smith 
and John Bush, ordinarily the addressee can not be 
specified. Since at step S910 the history information is 
referred to, however, it is judged that John to whom the 
document Contract was transmitted is John Smith, and 
the addressee is thus specified. At step S912 the doc- 
ument "Contract" is acquired from the filing database. 
At step 891 3 the document is transmitted to addressee 
John. Since there is notice instruction "Notify him by 
Phone" at step S914 it is determined that transmission 
of the notice is necessary. At step S915, as well as in 
Fig. 89, according to the rule of the knowledge base, an 
instruction is issued to a facsimile machine 901 as a re- 
ception apparatus to send a notice to a receiver by 
phone. At step S91 6 the other actions are not Instructed, 
and program control thereafter determined. 

Upon the receipt of the document, according to the 
notification instruction, the facsimile machine 901 noti- 
fies John by phone of the receipt of the document from 
Mr Doors. When the facsimile machine 901 can not no- 
tify the addressee by phone of the receipt of the docu- 
ment, the addressee may take Its place. 

[Thirty-first Embodiment] 

Fig. 95 is a flowchart showing example processing 
where, from the contents of a document that is input and 
analyzed, it is determined that a user should do some- 
thing, a query is actually transmitted to the user, and a 
required process is preformed automatically. Fig. 92 is 
a diagram showing an example input document. Fig. 93 
is a diagram showing the contents of a conversation be- 
tween a system and a user Fig. 94 is a diagram showing 
a document that is prepared by the system as the result 
of judgement of the conversation in Fig. 93. 

The processing for preparing an output document 
in Fig. 94 from an input document in Fig. 92 will be de- 
scribed while referring to the flowchart in Fig. 95. 

At step S950 the input document in Fig. 92 is ana- 
lyzed and the following objects are understood: (a) Mr 



John Smith arrives at Yokohama at three o'clock on No- 
vember 28; (b) Mr John Smith little understands Japa- 
nese; and (c) some one should meet him at the station. 

At step S951 what the user should do is determined 
from the contents that are obtained. Since it is deter- 
mined that some one must meet him at the station, pro- 
gram control advances to step S952. At step S952 a 
check is performed to determine whether a proxy (sys- 
tem) may execute the action. Since the system can not 
meet him at the station, program control goes to step 
S953. At step S953 a check is performed to determine 
whether an inquiry should be issued to the user, and if 
so, program control moves to step S954. In this exam- 
ple, since it is determined from the addressee of the in- 
put document that this is a request for Dr Aruna Rohra, 
program control moves to step S954. 

At step S954 a process for inquiring the user (Dr 
Aruna Rohra) is performed. In this case, as the result 
obtained at step S951 the user must meet John at the 
station, and an inquiry what to do is issued to the user 
At step S955 what to do next is determined from the re- 
sult of the inquiry, and whether or not an action is to be 
initiated is determined. In this example, the user in- 
structs to ask Mr Tanaka to meet him, the action is ini- 
tiated according to the instruction. Since there is a re- 
quest for Mr Tanaka to act as a proxy, it is assumed that 
the action must be initiated. At step S956 a document 
for requesting Mr Tanaka as a proxy is prepared and a 
contact is made with Mr Tanaka. In this example, an 
electronic mail document is prepared and transmitted. 
In this example, information that John arrives on 28th, 
an instruction for meeting him for the user and the orig- 
inal document that caused the request are attached to 
automatically form the document. 

As is described above, processing has been ex- 
plained for performing the matter (meeting John at the 
station) where from the input document the user must 
actually perfonm an action. 

[Thirty-second Embodiment] 

Fig. 96 is a diagram illustrating a thirty-second em- 
bodiment where a system is operated by a controller 

According to the system in this embodiment, devic- 
es and a controller directly or indirectly communicate 
with each other to specify the other 

Therefore, the following functions are provided: (1) 
automatic specifying of a device; (2) acquisition of a de- 
vice capability from the device; (3) a Ul (User Interface) 
inherent to a device; (4) operation by voice; (5) common 
basic operation for all the devices; (6) a Ul inherent to 
a user; and (7) a Ul having a high degree of freedom in 
consonance with the status. 

That Is, (1 ) only when a controller faces an operat- 
ing target device. It automatically recognizes the target 
device; (2) information for each device is acquired from 
the pertinent device via IrDA or via a wireless LAN; (3) 
a Ul inherent to each device is provided from the infor- 
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mation for the device; (4) an operation by voice can be 
performed; (5) common basic operation can be provided 
by using the same controller; (6) user identification in- 
formation is employed to acquire necessary information 
for each user from the database or the address book 
and to implement the Ul inherent to the user; and (7) the 
optimal Ul is provided in consonance with the status. 

In Fig. 96 since a controller 960 faces a facsimile 
machine 963, it acquires model identification informa- 
tion from the facsimile machine 963, and renders a Ul, 
which corresponds to the model, active. Then, following 
the displayed "Fax to" character string "Fax to John" is 
input w'rth a touch pen, and transmitted to the facsimile 
machine 963. John's facsimile number is read from an 
address book 965 in a personal computer 964= and a 
document set in the facsimile machine 963 or a docu- 
ment designated on a file in the personal computer 964 
is transmitted to the fax number The detailed process- 
ing will be described later 

[Thirty-third Embodiment] 

Fig. 97 is a diagram showing a status monitor 
In a system according to this embodiment, informa- 
tion of an apparatus that is other than the apparatus that 
a user directly operates can be referred to. 

Therefore, the following functions can be provided: 
(1 ) remote and handy management; (2) specifying of au- 
tomatic device; (3) common input/output for examining 
a status; (4) the same function as a portable telephone; 
(5) operation by voice; and (6) handling of all the devices 
that can input/output infrared rays. 

That is, (1) required information can be acquired 
even from a remote area from a target device; (2) only 
when a controller faces the target device, It can auto- 
matically recognize the target device; (3) common basic 
operation can be provided by using the same controller; 
(4) the usability is enhanced by providing the portable 
telephone function for the controller; (5) operation by 
voice can be performed; and (6) the common protocol 
is employed to cope with all the devices that can input 
and output infrared rays. 

In Fig. 97, a status monitor 970 selects a printer 961 , 
a copier 962 or a facsimile machine 963, and character 
string "Status" is input with a touch pen and instructed. 
The status monitor 970 retrieves status information from 
the device that the status monitor 970 faces, so that the 
user of the status monitor 970 can confirm the statuses 
of the devices. Even when a personal computer 964 is 
located at an area where a command from the status 
monitor 970 can not reach, the status of the personal 
computer 964 is requested to a printer 961 , which can 
communicate with the personal computer 964 via the 
network. Then, the status information can be retrieved 
to the status monitor 970 from the personal computer 
964 via the network and the printer 961 . 



[Thirty-fourth Embodiment] 

Fig. 98 is a diagram showing an example where a 
controller 960 recognizes a target model, and a corre- 

5 sponding Ul is read from the controller 960 and dis- 
played. The controller 960, which does not face a target 
device, has a default screen on which message "Select 
a device which you would like to use" is displayed, as is 
shown in the center in Fig. 98. 

10 The communication between the controller and the 
target model will now be described. It should be noted 
that (2) and (3) are not necessarily required. When, for 
example, in (4) a target model generates a signal to the 
controller at predetermined intervals, the same effect 

15 can be obtained. The positional relationship between 
the controller 960 and a target mode! may be employed 
to determine that the controller 960 selects the target 
model- Further, in (4) the controller 960 may identify a 
target model by examining the shape of the model, or 

20 by reading the bar code attached to the target model, in 
other words, only the essential portion is shown in Fig. 
98. 



(1 ) A user directs the controller 960 to a target mod- 
25 el 

(2) The usertouches a touch panel on the controller 
960 in order to determine the timing at which the 
controller 960 transmits to the target model a target 
model identification signal request signal. Or, the 

30 controller 960 is set in advance to transmit a target 
model identification signal request signal at a pre- 
determined interval. 

(3) The controller 960 transmits a target model iden- 
tification signal request signal to the target model. 

35 (4) A target model identification signal is transmitted 
upon the receipt of the target model identification 
signal request signal from the controller 960, or ac- 
cording to the setup that a target model identifica- 
tion signal request signal at a specified Interval. 

40 (5) The controller 960 that receives the target model 
identification signal determines a Ul in consonance 
with a target model indicated by the signal, and dis- 
plays a corresponding Ul internally stored. In this 
case, the identification signal is provided for each 

45 model; however, when an identification signal is 
provided tor each device, a different Ul can be em- 
ployed tor each of devices though they are the same 
model. 

50 Fig. 99 is a flowchart showing the processing per- 
formed by the controller 960 when the controller recog- 
nizes a target model, and reads and displays a corre- 
sponding Ul. The processing performed by the control- 
ler 960 will now be described. 

55 At step S991 the controller 960 waits until an in- 
struction from a user is Input. At step S992 a target mod- 
el identification signal request signal is transmitted by 
an infrared ray communication method in order to obtain 
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a target model identification signal from the target mod- 
el At step S993 the controller 960 waits until the target 
model outputs a target model identification signal. At 
step S994, the Ul corresponding to the received target 
model identification signal is acquired from internal 
memory and is displayed. 

Fig. 100 is a flowchart showing the processing per- 
formed by a target model when the controller 960 rec- 
ognizes a target model, and reads and displays a cor- 
responding Ul. The processing performed by the target 
mode! will now be described. 

At step SI 001 , the target model waits for a request 
signal from the controller 960. At step S1002 the target 
model transmits a target model identification signal by 
an infrared communication method, etc. 

|Thirty-titlh Embodiment] 

Fig 101 IS a diagram showing an example where a 
coniroller 950 receives a Ul from a target model and dis- 
plrtyb il The conlioller 960, which does not face a target 
device has a default screen, as is shown in the center 
in Fig 101 

The communication between the controller and the 
target model will now be described It should be noted 
that (2) and (3) are not necessarily required. When, for 
example, in (4) a target model generates a signal to the 
controller a! predetermined intervals, the same effect 
can be obtained The positional relationship between 
the controller 960 and a target model may be employed 
to determine that the controller 960 selects the target 
model Further in (4) the controller 960 may identify a 
target model by examining the shape of the model, or 
by reading the bar code attached to the target model. In 
other words, only the essential portion is shown in Fig. 
101. 

(1 ) A user directs the controller 960 to a target mod- 
el. 

(2) The user touches a touch panel on the controller 
960 in order to determine the timing at which the 
controller 960 transmits a Ul request signal to the 
target model. Or the controller 960 is set in advance 
to transmit a Ul request signal at a predetermined 
interval. 

(3) The controller 950 transmits a Ul request signal 
to the target model. 

(4) A Ul signal is transmitted upon the receipt of the 
Ul request signal from the controller 960, or accord- 
ing to the setup that a Ul request signal at a speci- 
fied interval. 

(5) The controller 960 that receives the Ul signal dis- 
plays the Ul. 

Fig. 102 is a flowchart showing the processing per- 
formed by the controller 960 when the controller re- 
ceives a Ul from a target model and displays it. The 
processing performed by the controller 960 will now be 



described. 

At step S1021 the controller 960 waits until an in- 
struction from a user Is input. At step S1 022 a Ul request 
signal is transmitted by an infrared ray communication 

5 method in order to obtain a Ul signal from the target 
model. At step SI 023 the controller 960 waits until the 
target model outputs a Ul signal, at step SI 024, the re- 
ceived Ul is displayed. 

Fig. 103 is a flowchart showing the processing per- 

10 formed by a target model when the controller 960 re- 
ceives a Ul from a target model and displays it. The 
processing performed by the target model will now be 
described. 

At step SI 031 , the target model waits for a request 
75 signal from the controller 960. At step 81032 the target 
model transmits a Ul signal by an infrared communica- 
tion method, etc. 

[Thirty-sixth Embodiment] 

20 

Fig. 104 is a diagram showing an example where 
Uls of devices are stored in a wireless LAN server 1041 
and when a controller 960 faces a copier 962 as a target 
model and requests a Ul, a Ul is transmitted, upon a 
2S request from the copier 962, from the wireless LAN serv- 
er 1041 to the controller 960, and is displayed. 

The controller 960, which does not face a target de- 
vice, has a default screen, as is shown in the center in 
Fig. 104. 

30 The communication between the controller and the 
target model will now be described. 

(1 ) A user directs the controller 960 to a target mod- 
el (a copier 962). 

3S (2) The user touches a touch panel on the controller 
960 in order to determine the timing at which the 
controller 960 transmits a Ul request signal to the 
target model. Or, the controller 960 is set in advance 
to transmit a Ul request signal at a predetermined 

40 interval. 

(3) The controller 960 transmits controller identifi- 
cation data (controller A) to identify itself and a Ul 
request signal to the target model. 

(4) When the target model receives the controller 
4S identification data and the Ul request signal from the 

controller, the target model transmits, to the server 
1041 thai manages the target model, the controller 
identification data (controller A) and target model 
identification data (copier) to identify itself, and re- 
so quests the transmission of the Ul. 

(5) The server 1041 transmits the Ul of the target 
model, which is indicated by the target model data, 
via a wireless LAN to the controller 960, which is 
indicated by the controller identification data. 

55 (6) The controller 960 receives the Ul addressed to 
itself, and displays the Ul. 

Fig. 105 is a flowchart showing the processing per- 
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tormed by the controller 960 when the controller re- 
ceives a Ul via the wireless LAN. instead of directly re- 
ceiving it from a target model, and displays it. 

The processing performed by the controller 960 will 
now be described. At step SI 051 the controller 960 
waits until an instruction from a user is input. At step 
SI 052 a U! request signal for requesting a Ul and con- 
troller identification data for identifying a controller are 
transmitted to the target model by an infrared ray com- 
munication method. At step SI 053 the controller 960 
waits until a Ul signal is output by any device. At step 
SI 054, the received Ul is displayed. 

Fig. 106 is a flowchart showing the processing per- 
formed by the target model when the controller receives 
a Ul via the wireless LAN, instead of directly receiving 
it from a target model, and displays it. The processing 
performed by the controller 960 will now be described. 

At step S1061 the controller 960 waits until an in- 
struction from a user is input. At step 81062 controller 
identification data received from the controller 960 and 
the target model identification data are transmitted to 
the server 1041 , and issuance of the Ul is requested. 

Fig. 107 is a flowchart showing the processing per- 
formed by the server 1041 when the controller receives 
a Ul via the wireless LAN, instead of directly receiving 
it from a target model, and displays it. The processing 
performed by the server 1041 will now be described. 

At step S1071 the server 1041 waits until a Ul trans- 
mission request signal is transmitted from the target 
model. At step SI 072 the Ul of the model, which is in- 
dicated by the target model identification data, is trans- 
mitted to the controller 960, which is indicated by iden- 
tification data received from the target model. 

[Thirty-seventh Embodiment] 

Fig. 108 is a diagram showing an example where 
Uls of devices are stored in a wireless LAN server 1 041 
and when a controller 960a or 960b faces a facsimile 
machine 963 as a target model and requests a Ul, a dif- 
ferent Ul for each user is transmitted, upon a request 
from the facsimile machine 963, from the wireless LAN 
server 1041 to the controller 960a or 960b, and is dis- 
played. 

The controller 960a or 960b, which does not face a 
target device, has a default screen, as is shown in the 
center in Fig. 108. 

The communication between the user, the control- 
ler 960a or 960b and the target model wilt now be de- 
scribed. 

(1) A user directs the controller 960a or 960b to a 
target model. 

(2) The user touches a touch panel on the controller 
960a or 960b in order to determine the timing at 
which the controller 960a or 960b transmits a Ul re- 
quest signal to the target model. Or, the controller 
960a or 960b is set in advance to transmit a Ul re- 



quest signal at a predetermined interval. 

(3) The controller 960a or 960b transmits controller 
identification data (controller A or controller B) to 
identify itself, user identification data (a senior or a 

5 beginner) for identifying the user, and a Ul request 
signal to the target model. 

(4) When the target model receives, from the con- 
troller 960a or 960b, the controller identification da- 
ta, the user identification data and the Ul request 

10 signal, the target model transmits, to the server 
1041 that manages the target model, the received 
controller identification data, the user identification 
data and the target model identification data to iden- 
tify itself. 

is (5) The server 1041 transmits the Ul of the target 
model, which is indicated by the user identification 
data and the target model data, via a wireless LAN 
to the controller 960a or 960b, which is indicated by 
the controller identification data. In the example in 

20 Fig. 108, a Ul that is used for multi-address com- 
munication, which is difficult for beginners, is trans- 
mitted only to seniors, not to beginners. 
(6) The controller 960a or 960b receives the Ul ad- 
dressed to itself, and displays the Ul. 

25 

In the above example, the controller transmits the 
level data for a user is transmitted as user identification 
data. However, the controller may transmit information, 
such as the names of users, to identify individual users. 

30 and the target device or the server may identify the level 
of a user from the received identification data. An inde- 
pendent Ul may be prepared for each user 

The above example has been employed only for the 
UL The contents of urgent mail addressed to the user 

35 or the schedule to be Immediately informed are collect- 
ed in, for example, the server, and they may be included 
in the Ul while the user identification data that differs 
among the users is referred to. and the resultant data 
can be transmitted to the controller 

40 Fig. 109 is a flowchart showing the processing per- 
formed by the controller 960 when the controller re- 
ceives a different Ul for each user, via the wireless LAN, 
instead of directly receiving it from a target model, and 
displays it. The processing performed by the controller 

45 960 will now be described. 

At step SI 091 the controller 960a or 960b waits until 
an instruction from a user is input. At step SI 092 a Ul 
request signal for requesting a Ul. controller identifica- 
tion data for identifying a controller, the user identifica- 

so tion data for identifying the user are transmitted to the 
target model by an infrared ray communication method. 
At step S1093 the controller 960a or 960b waits until a 
Ul signal is output by any device. At step SI 094. the 
received Ul is displayed. 

55 Fig. 110 is a flowchart showing the processing per- 
formed by the target model vsrtnen the controller receives 
a different Ul for each user, via the wireless LAN, instead 
of directly receiving it from a target model, and displays 
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it. The processing performed by the controller 960a or 
960b will now be described. 

At step S1 1 01 the controller 960a or 960b waits until 
an instruction from a user is input. At step S1102 the 
controller identification data received from the controller 
960, the user identification data, and the target model 
Identification data are transmitted to the server 1041, 
and the issuance of the Ul is requested. 

Fig. Ill is a flowchart showing the processing per- 
formed by the server 1041 when the controller receives 
a different Ul for each user via the wireless LAN, instead 
of directly receiving it from a target model, and displays 
it. Fig. 112 is a table showing Uls that are determined 
by the target model and user identification data. The 
processing performed by the server 1041 will now be 
described. 

At step S11 1 1 the server 1041 waits until a Ul trans- 
mission request signal is transmitted from the target 
model. At step S111 2 in accordance with the definitions 
in Fig. 112, the Ul that corresponds to the target mode! 
identification data and the user identification data is 
transmitted to the controller 960a or 960b, which is in- 
dicated by identification data received from the target 
model. 

[Thirty-eighth Embodiment] 

Figs. 113 and 114 are diagrams illustrating an ex- 
ample where provided is a function that a model to be 
operated in front does not have. The communication be- 
tween a user, a controller and a target model will now 
be described. 

(1 ) A user faces a controller 960 to a target model, 
and designated a desired function. 

(2) The target model receives a request from the 
controller 960 and transmits it to a server 1041 . 

(3) The server 1041 transmits a Ut of a model that 
corresponds to the requested function. 

(4) The controller 960 displays a received UL 

(5) The user performs operation according to an in- 
struction of the UL In this example, the user sets a 
document to be transmitted to a copier 962 in front, 
and the controller 960 instructs the destination to 
execute the operation. 

(6) The target model handles what it can do and re- 
quests another apparatus to process what it can not 
do. In this case, since the copier 962 reads a doc- 
ument but can not facsimile the document, it trans- 
mits to a facsimile machine 963 the document data 
and the contents of the operation instruction. 

(7) The facsimile machine 963 transmits by fax the 
received document data to the instructed destina- 
tion. 

In the above example, to determine an apparatus 
that the copier 962 asks for the function that the copier 
962 can not perform, each apparatus may have knowl- 



edge for other apparatuses. Or, the apparatuses on the 
network may be queried if they can execute the function, 
and according to the result, the apparatus to be asked 
for may be determined. Or, an inquiry may be issued to 
5 the sen/er 1041 that has knowledge of the models that 
correspond to functions. 

[Thirty -ninth Embodiment] 

10 Fig. 115 is a diagram illustrating an example where, 
when a controller can not directly communicate with a 
target model in front because an obstacle is present in 
route between the target model and the controller, indi- 
rect communication is attempted using another route. 

15 The communication between the user, the control- 
ler and the target model will now be described. It should 
be noted that (2) and (3) are not necessarily required. 
When, for example, in (4) a target model generates a 
signal to the controller at predetermined intervals, the 

20 same effect can be obtained. In other words, only the 
essential portion is shown in Fig. 115. 

(1 ) A user directs the controller 960 to a target mod- 

25 (2) The user touches a touch panel on the controller 
960 in order to determine the timing at which the 
controller 960 transmits a Ul request signal to the 
target model. Or, the controller 960 is set in advance 
to transmit a Ul request signal at a predetermined 

30 interval 

(3) The controller 960 transmits a Ul request signal 
to the target model. 

(4) A Ul signal is transmitted upon the receipt of the 
Ul request signal from the controller 960, or accord- 

35 ing to the setup that a Ul request signal at a speci- 
fied interval. 

(5) The controller 960 that receives the Ul signal dis- 
plays the Ul. 

(6) If a Ul is not transmitted following a predeter- 
40 mined period of time after the Ul request signal was 

transmitted, or if a Ul is not transmitted during a pe- 
riod longer than a predetermined interval, whbh is 
set for transmission of Uls, another route is em- 
ployed to transmit a Ul request signal. In this exam- 
45 pie, when infrared communication can not be per- 
formed, communication through a wireless LAN is 
attempted. 

(7) The Ul acquired across the LAN is displayed, 

50 Fig. 1 1 6 is a flowchart showing the processing per- 

formed by a controller for an example where, when the 
controller can not directly communicate with a target 
model in front because an obstacle is present in route 
between the target model and the controller, indirect 

55 communication is attempted using another route. The 
processing performed by the controller 960 will now be 
described. 

At step S1 161 the controller 960 waits until an in- 
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struclion from a user is input. At step S11 62 a Ul request 
signal tor requesting a Ut and controller identification da- 
ta for identifying a controller are transnnitted to the target 
model by an infrared ray communication method. At 
step S1163 a check is performed to determine whether 
or not the Ul has been received. At step S1 1 64 a check 
is performed to determine whether a predetermined pe- 
riod of time has elapsed following the transmission of 
the request. At step S1165 a Ul request signal, for re- 
questing a Ul, and controller identification data, for iden- 
tifying a controller, are transmrtted by a route, such as 
across a wireless LAN, that is different from the route at 
step S1162. At step S1 166 the received Ul is displayed. 

In this case, the Ul request signal that is directly 
transmitted to the target model is to be sent to the wire- 
less LAN server 1041 . However, since the wireless LAN 
server 1041 that receives the Ul request signal is not 
sure of a target model, first a menu for selecting a model 
(function) is displayed. AUI corresponding to the select- 
ed model is displayed. Upon receipt of the model selec- 
tion signal from the controller 960, the wireless LAN 
server 1041 may transmit to the controller 960 a Ul that 
corresponds to the model, or may transmit the menu and 
a plurality of Uls available from the menu. 

[Fortieth Embodiment] 

Fig. 117 is a diagram showing an example where 
an operation and an action performed by a user, the lime 
and associated important information are stored as his- 
tory. 

An explanation will be given tor the processing for 
storing, as history, the operatiorVaction periormed by 
the user, the time and the associated important informa- 
tion. 

(1) A user sets a document in a facsimile machine 
963, and Instructs the transmission of the document 
and the user's name (sender's name) to Mr. a ot A 
company. 

(2) According to the instruction, the facsimile ma- 
chine 963 transmits the document to Mr. a. Also ac- 
cording to the instruction from Tom, the facsimile 
machine 963 notifies the server 1 041 of the fact that 
the document has been transmitted to Mr. a togeth- 
er with the transmitted document and the transmis- 
sion time. 

(3) The server 1041 saves the received document 
as a file "File ABC" and stores the fact of the receipt 
as history information. 

Fig. 1 18 is a flowchart showing the processing per- 
formed by the controller for an example where an oper- 
ation and an action performed by a user, the time and 
the associated important information are stored as his- 
tory data. The processing performed by the controller 
will now be described. 

At step S1 181 the controller 960 waits until an in- 



struction from a user is input. At step 81182 a request 
signal, for the instruction of the user, controller identifi- 
cation data, for identifying a controller, and the user 
identification data, for identifying the user, are transmit- 
5 ted by an infrared ray communication method. At step 
S1 1 83 the controller 960 waits until a Ul signal is output 
by any device. At step S1184. the received Ul is dis- 
played. 

Fig. 119 is a flowchart showing the processing per- 
10 formed by the target model for an example where an 
operation and an action performed by a user, the time 
and the associated important information are stored as 
history data. The processing performed by the target 
model will now be described. 
15 At step S1 191 the target model waits until a request 
signal is transmitted by the controller 960. At step S11 92 
a process corresponding to the request signal from the 
controller 960 is performed. In the example shown in 
Fig. 117 a document set by the user is read and trans- 
20 mitted to Mr. a of A company. At step S1 193 controller 
identification data received from the controller 960, the 
user identification data, and target model identification 
data, and important data associated with the action that 
was performed, are transmitted to the server 1401. In 
25 the example in Fig. 1 1 7, according to the instruction from 
Tom, the fact that the document was transmitted to Mr. 
a is sent to the server 1 041 , together with the transmis- 
sion time and the transmitted document that is the im- 
portant information associated with the fact. 
30 Fig. 1 20 is a flowchart showing the processing per- 

formed by the server for an example where an operation 
and an action performed by a user, the time and the as- 
sociated important information are stored as history da- 
ta. Fig. 121 is a diagram showing example history infor- 
ms matron that is updated as the result of processing. The 
processing performed by the server will now be de- 
scribed. 

At step SI 201 the server waits until it receives a 
request signal from the target model. At step SI 202 the 
40 important information associated with the received fact 
is stored. In the example in Fig. 117 the document the 
transmission of which is instnjcted by Tom is saved as 
File ABC. 

At step SI 203 the history data are updated in con- 
45 sonance with the received fact. In the example shown 
in Fig. 117 the fact that Tom transmitted a document to 
. M. a of A company at 10:00 on July 5, 1 996 is recorded 
together with the file name stored at step 81 201. Fig. 
1 21 is a table showing updated history data; the fact that 
50 Mary printed ten copies of File XYZ on July 3, 1 996. and 
the fact that Tom transmitted File ABC to Mr. a of A com- 
pany at 10:00 on July 5, 1996. At step 81 204 a Ul that 
indicates the normal termination of the processing is 
transmitted to the controller that is defined by the con- 
55 troller identification data received from the target model. 
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[Forty-first Embodiment] 

Figs. 122 and 123 are diagrams showing an exam- 
ple where an operation/action performed by a user, the 
time and associated important information are stored as 
history data and based on the history data an operation 
is performed. 

An explanation will be described the processing for 
the example where an operation/action performed by a 
user, the lime and associated important information are 
stored as history data and based on the history data an 
operation is performed. 

(1 ) A controller 960 instructs a copier 962 to show 
the history of Tom. 

(2) Upon receipt of the instruction, the copier 962 
asks a server 1 401 tor managing the history to show 
the history of Tom. 

(3) The server 1041 refers to the history data and 
transmits, as a Ul, only the history associated with 
the Tom to the copier 962. The copier 962 transmits 
the received history data to the controller 960. 

(4) Tom selects a specific fact from the history data 
displayed on the controller 960, and instructs an ac- 
tion. In this case, since Tom needs a material that 
was previously sent to A company, the fact "File 
ABC was sent to Mr. a of A company" is selected 
and action "copy" is instructed to acquire the mate- 
rial from the copier 962 in front. 

(5) The copier 962 transmits to the server 1041 the 
instruction "copy File ABC* from the user. 

(6) The server 1041 sends a corresponding docu- 
ment "File ABC" to the copier 962. 

(7) The copier 962 prints a document received by 
the server 1041 . 

In the above example, the fact "File ABC was sent 
to iVIr. a of A company" is selected from the history in 
order to acquire necessary material from the copier 962 
in front. When the name of document "File ABC" to be 
output is obtained, the list of files can be displayed in- 
stead of the history data and the file name can be se- 
lected, or the file name can be input directly. 

In addition, in the above example, the document to 
be extracted is the one that is transmitted by the facsim- 
ile machine 963 and stored in the server 1041 . A docu- 
ment stored in a PC across the network can also be em- 
ployed. 

Fig. 124 is a diagram showing the structure of the 
controller 960 used in this embodiment. 

A display 1241 is used to display data on a liquid 
crystal screen. A resistance film is attached to the dis- 
play 1 241 , and data can be input with a pen 1 242. A 
button on a screen may be selected by the pen 1 242. 
Furthermore, a character recognition function is provid- 
ed, and the characters written with the pen 1242 can be 
input. Instead of this function, a voice input/recognition 
function may be employed to input information. 



The input information can be transmitted from a 
communication unit 1243 to another apparatus A CPU 

1244 for controlling and a storage unit 1245 are provided 
inside the apparatus, and Input information and informa- 

5 tion received by the communication unit 1243 and the 
history of the operations are stored. The storage unit 

1245 can be a SRAM card, an SIMM or an HDD. The 
controller 960 is battery operated and water resistant 

Fig. 125 is a diagram illustrating another example 

10 structure of the controller 960 used in the embodiment. 
A display 1251 is used to display data using liquid 
crystal or a CRT. A touch panel can be additionally pro- 
vided as an option to the display 1 251 . Thus, Information 
can be input by a finger or a pen. An input unit 1252 is 

IS used to input information by manipulation of a button. 
The input data can be transmitted from a communication 
unit 1253 to another apparatus. A CPU 1254 for control- 
ling and a storage unit 1255 are provided inside the ap- 
paratus, and input information and information received 

20 by the communication unit 1253 and the history of the 
operations are stored. The storage unit 1 255 can be a 
SRAM card, an SIMM or an HDD. 

The controller in Fig. 1 25 can be used as a button 
controller with the storage unit 1 255 being removed. The 

2S controller can be used as a personal digital assistant 
(PC) with the communication unit 1253 being removed. 
In addition, the controller can be used as is shown in 
Fig. 124, with the input unit 1252 being removed and 
with the touch panel provided. 

30 Fig. 126 is a diagram showing an additional exam- 
ple structure of the controller 960 used in this embodi- 
ment. 

A display 1261 is used to display data using liquid 
crystal. An input unit 1262 is used to input information 

3S by manipulating a button. The input data can be trans- 
mitted by an infrared communication unit 1263 to anoth- 
er apparatus. A CPU 1264 for controlling and a storage 
unit 1265 are provided inside the apparatus, and input 
information and information received by the infrared 

40 communication unit 1 263 and the history of the opera- 
tions are stored. The storage unit 1265 can be a SRAM 
card, an SIMM or an HDD. Furthermore, a PCMCIA slot 
1266 is provided, and, for example, a PCMCIA card is 
loaded into the slot 1 266 to enable the connection by a 

45 PCMCIA bus, so that two communication channels can 
be employed. The infrared communication unit 1263 
may be used only tor transmission by light emission, and 
the PCMCIA bus may be used for reception. Of course, 
the connection by the PCMCIA bus may not be em- 

50 ployed. 

[Forty-second Embodiment] 

This system arrangement according to this embod- 
55 iment is as shown in Figs. 41 A and 41 B, and information 
processing apparatuses 411 , 412 and 413 can commu- 
nicate with each other. 

When the information processing apparatus 411 
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needs information stored in the information processing 
apparatus 412, or when the information is to be trans- 
mitted to the information processing apparatus, or when 
an instruction is to be transmitted to the information 
processing apparatus 412, the information processing 
apparatus 411 requests the communication with the in- 
formation processing apparatus 412 and exchanges re- 
quired information and an instruction. 

Any apparatus can be used for the information 
processing apparatuses 41 1 , 41 2 or 41 3 so long as they 
have necessary communication function; a printer, a 
scanner, a facsimile machine, a desktop PC or a digital 
camera can be used. 

Fig. 127 is a diagram illustrating the functional ar- 
rangement for this embodiment. 

An operating unit 41 displays a screen required for 
a user to operate in accordance with data stored in an 
operation data memory area 42. An operation per- 
formed by a user or an instruction issued by a user is 
acquired by the operation unit 41 and transmitted to a 
task reception unit 43. 

The operation data 42 is data to be referred to by 
the operation unit 41, and specifies the process at the 
operation unit 41 . In this embodiment, based on the in- 
formation received by the task reception unit 43, the op- 
eration data 42 is altered or updated by a process exe- 
cution unit 47, which will be described later, so that flex- 
ible procedures can be provided. 

One of methods for operating an apparatus other 
than the information processing unit 411, i.e., the infor- 
mation processing apparatus 412, is the one for acquir- 
ing the operation data 42 for the information processing 
apparatus 412 and for altering and updating the opera- 
tion data 42 of the information processing apparatus 
411. 

In another example, the presence of the information 
processing apparatus 412, which is connected across 
the network to the informatbn processing apparatus 411 
in this embodiment, is detected, so that a new function 
obtained by combining the two information processing 
apparatuses can be Implemented. Information required 
for an Instruction for the function is added to the opera- 
tion data 42, and thus, the function obtained by combin- 
ing a plurality of apparatuses can be provided. 

The task reception unit 43 receives not only Infor- 
mation from the operation unit 41 , but also receives, as 
the status change, a change in the environment o1 the 
information processing apparatus 411 or a task received 
from another apparatus, such as the information 
processing apparatus 412. Task reception unit 43 also 
detects the connection of a new information processing 
apparatus to the network, receives as a task a print in- 
struction from another apparatus, or detects that no 
process is performed during a specified period of time. 
As a result, received information is added as a new task 
to a task table 44. 

The task reception unit 43 is required as well as the 
process execution unit 47, which will be described later, 



to communicate with different apparatuses. 

The task table 44 is a table in which are stored tasks 
having the activation condition and the contents. Figs. 
129A to 129F are examples for the task table 44. 
5 In Figs. 1 29 A to 1 29F, the tasks are stored from the 

top of the table in the order of priorities. The addition of 
a task to the task table 44 is performed by the previously 
mentioned task reception unit 43 and the process exe- 
cution unit 47 to be described later. Extraction of the task 
10 from the task table 44 is performed by a task analysis 
unit 45, which will be described later. 

The task analysis unit 45 extracts a task having the 
highest priority from the task table 44, and analyzes the 
object of the task. Since the task analysis unit 45 ana- 
'is lyzes the object, a planning decision unit 46 and the 
process execution unit 47 can avoid inefficient process- 
ing and can perform an optimal process. 

For example, a task 63 in the task table 44 in Figs. 
1 29A to 1 29F is analyzed and understood that the direct 
20 object is printing and the final object is to transmit a print- 
ed document to a user. If the final object can be achieved 
by an efficient method, it is found that even printing is 
unnecessary, and planning is performed according to 
that effect. The information to be processed is also an- 
2S alyzed, and it isfound that format conversion is required. 

The planning decision unit 46 receives the analysis 
from the task analysis unit 45, makes planning for a 
process to be performed. For example, for the task 63 
stored in the task table 44 in Figs. 1 29A to 1 29F a proc- 
30 ess for extracting an object from an apparatus B before 
printing of the object is planned. 

The process execution unit 47 performs a process 
according to the contents determined by the planning 
decision unit 46, e.g., sends an instruction to another 
35 apparatus. The process execution unit 47 will be de- 
scribed in detail in a forty-third embodiment and follow- 
ing embodiments. The functional arrangement in Fig. 
127 can be provided by the hardware configuration in 
Fig. 3. 

40 Fig. 128 is a flowchart showing the processing for 
this embodiment. 

When the information processing apparatus 41 1 in 
this embodiment is activated, at step S51 the task table 
44 is initialized as is denoted by 61 in Fig. 129A. 

45 Then, at step S52 a check is performed to deter- 
mine whether the task reception unit 43 fias received, 
as the status change, information that is acquired by the 
operation unit41 for an operation instruction from a user, 
an environmental change^ or a task from another appa- 

so ratus. When the task is received, program control ad- 
vances to step S53, and as is shown in Fig. 129B, the 
analysis task 62 of the received information is added to 
the head of the task table 44. 

Following this, at step S54 a check is performed to 

55 determine whether or not an executable task is present 
in the task table 44. If there is no such task, program 
control returns to step S52, whereat the reception of the 
task is examined again. If there is such a task, program 
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control moves to step S55, whereat the task having the 
highest priority is extracted from the task table 44. 

At step S56 the object of the task is analyzed, and 
at step S57 the process to be performed is planned and 
decided. If the task table is in the state shown in Fig. 
129B, highest priority task 62 "ANALYZE" is extracted, 
it is apprehended that its object is analysis of received 
information, and planning for analysis is performed. 

At step S58 the process according to the planning 
is performed, and as needed, a new task is added to the 
task table 44. When, for example, the received informa- 
tion is analyzed according to the plan obtained from the 
task 62 "ANALYZE" in Fig. 129B, the contents "Print 
'contract.doc* from B" which is instructed by a user 
through the operation unit 41 , are analyzed and a new 
task 63 "PRINT" Is added as is shown in Fig. 129C. 

Program control returns again to step S52. Howev- 
er, since no new task is received, program control 
moves to step S54, and the above process is repeated. 
As a result, task 63 "PRINT" in Fig. 129C, which was 
added at step S58, is extracted, analyzed and planned. 
To execute task PRINT it is found that the object must 
be present in the information processing apparatus 411 . 
and new task 64 "PULL" and task 65 "PRINT" which em- 
ploys the "pulled" object, are added as is shown in Fig. 
129D. 

In addition, when the processing is repeated, the 
task 64 "PULL" in Fig. 129D is executed- Since the con- 
tents of the task PULL are important portions for the 
present invention, detailed explanation for them will be 
given for the forty-third embodiment and the following. 

When the process is further repeated, the object 
that is acquired as the result of the task PULL is em- 
ployed to perform task 65 in Fig, 129E. As a result, the 
task table 44 is returned to the initial state, as is shown 
in Fig. 129F 

[Forty-third Embodiment] 

Fig. 130 is a diagram showing an example input/ 
output device that can operate another apparatus. 

The operation procedure (Ul) implemented by the 
operation unit 41 is performed by referring to the oper- 
ation data 42 that the information processing apparatus 
41 1 originally includes, or by referring to the operation 
data 42 that is acquired from another information 
processing apparatus 412 connected to the network. 

An instruction 71 by a user's voice, or the employ- 
ment of an operation panel screen 72 to designate a tar- 
get device, is performed by referring to the operation da- 
ta 42 that are originally included in a copier, which 
serves as the information processing apparatus 411. 

The cqpier 74, which is the information processing 
apparatus 41 1 , executes the task PULL at the process 
execution unit 47, and refers to on the operation data 
42, which are acquired from a facsimile machine 75 that 
serves as the information processing apparatus 412 
connected via the network, to designate a transfer des- 



tination and a condition by using the screen 73 of the 
operation panel. 

The operation data 42 of the facsimile machine 75 
may be held by the copier 74 in advance, and be referred 
s to as needed. 

The operation data 42 of the facsimile machine 75 
may be acquired from another device, such as a server 

A specific method for communication between the 
information processing apparatuses 41 1 and 41 2 will be 
70 explained in detail for a forty-ninth embodiment and the 
following embodiments. 

[Forty-fourth Embodiment] 

75 Fig. 131 is a diagram showing an input/output ap- 
paratus that can operate a function obtained by a com- 
bination of a plurality ot devices. 

The procedure (Ul: User Interface) implemented by 
the operation unit 41 is performed by referring to the op- 

20 eration data 42 the information processing apparatus 
411 originally includes, or by referring to the operation 
data 42 that is updated as the result of the detection of 
the infornnation processing apparatus 412 connected 
via the network. 

^5 The number of copies using the operation panel 81 

in Fig. 131 is designated by referring to the operation 
data 42 that is originally included in the copier 74, which 
serves as the information processing apparatus 411. 
The "Copy" displayed in the lower portion of the opera- 
te tion panel indicates an instruction of copying, and 
"transmit" displayed in the hatched state indicates that 
the transmission process is disabled. 

As the result of the detection of the facsimile ma- 
chine 75, which serves as the information processing 

35 apparatus 41 2 connected via the network, the operation 
data 42 of the copier 74 are updated to the function that 
can be used, i.e., the transmission function from which 
the hatching is removed, and is referred to, so that the 
number of copies on a display 82 is designated using 

40 the operation panel, as well as the operation panel 81 . 

\Nhen a user selects and instructs the execution of 
the transmission function, a display 83 for designating 
the transmission destination is provided and the trans- 
mission function can be executed. As is described 

^5 above, when a user need only set a desired document 
on a copier near the user and instruct the execution of 
the transmission function, the document can be trans- 
mitted by using the function of the facsimile machine 75, 
which is at a remote area and is connected via a net- 

50 work. This accomplishes both the document input func- 
tion of the copier and the document transmission func- 
tion of the facsimile machine. 

When the facsimile machine 75 is not connected on 
the display 81 , the transmission operation on the display 

55 83 for designating the transmission destination is per- 
formed in advance, and at the same time that the con- 
nection of the facsimile machine 75 is detected, the con- 
tents of the transmission operation is transmitted to the 
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facsimile machine 75. Thus, in consonance with the 
contents, the automatic transmission can be performed, 
or the reservation operation, such as a confirmation to 
a user before automatic transmission, as is shown in a 
screen 84, can be performed. 

In this embodiment, while the facsimile machine 75 
is not connected, the display "transmit" for a transmis- 
sion instruction is displayed by hatching to indicate 
transmission is enabled. Such a display may not be pro- 
vided at all 

A specific method lor communication between the 
information processing apparatuses 411 and 41 2 will be 
described in detail for the forty-ninth embodiment and 
the following 

[Forty-tilth Embodiment | 

Fig. 132 IS H dtHgrnm illustrating an example input/ 
output device thai c^n communtcaic with a user. 

In the example in F«g \ 32 a user sends an instruc- 
tion 93 to a copiet 91 Itint ib pfiy;>icnliy kxated in a re- 
mote area in order to permit a My Desk 92, a desktop 
personal computer to display the current location of the 
user. 

The instruction 93 'Display prcGont place on 
<MyDesk>V which is sent to the copter 91 that serves 
as the information processing apparatus 411, is trans- 
mitted from the operation unit 41 to the task reception 
unit 43, The contents are analyzed and planning is per- 
formed on the results Then, the execution unit 47 exe- 
cutes the final data as an instruction task INDICATE for 
a PC 92, which serves as the information processing 
apparatus 412 connected via the network. 

The PC 92, which is the information processing ap- 
paratus 412 connected via the network, receives conn- 
munication task COMMUNICATION from the reception 
unit 43, analyzes a received instruction, and perform 
planning on the results. Then, the execution unit 47 dis- 
plays a message on a screen. 

A specific method for communication between the 
information processing apparatuses 41 1 and 41 2 will be 
described in detail for the forty-ninth embodiment and 
the following. 

[Forty-sixth Embodiment] 

Fig. 133 is a diagram illustrating an example input/ 
output device that can acquire Information from another 
apparatus. 

In the example in Fig. 1 33, a user sends an instruc- 
tion 1333 to a printer 1 331 that is located in a physically 
remote area to print information 1 334 "contract. doc" that 
is stored in apparatus B1 332. 

The instruction 1333 "Please fetch 'contract. doc' 
from B and print it" which the user sent to the printer 
1331 that serves as the information processing appara- 
tus 411 , is transmitted from the operation unit 41 to the 
task reception unit 43. The contents are analyzed and 



planning is performed on the results. Then, the execu- 
tion unit 47 executes the final data as information acqui- 
sition task PULL for the apparatus B 1332, which serves 
as the information processing apparatus 41 2 connected 

s via the network. The apparatus B can be a printer, a 
scanner, a desktop PC or a filing server 

The apparatus B1 332, which serves as the informa- 
tion processing apparatus 41 2 connected via the net- 
work, receives the information acquisition task PULL 

10 from the reception unit 43, analyzes the received in- 
struction, and performs planning on the data. The exe- 
cution unit 47 then transmits corresponding information 
1334 "contract.doc". 

A specific method for communication between the 

15 information processing apparatuses 41 1 and 41 2 will be 
described in detail for the forty-ninth embodiment and 
the following. 

[Forty-seventh Embodiment] 

20 

Fig. 1 34 is a diagram illustrating an example input/ 
output device that can be stored in another apparatus. 

In the example shown in Fig. 134. a user reads a 
document 115 using a scanner 111 and sends an in- 
2S struction 11 3 to permit a filing sen/er 112 at a physically 
remote area to perform filing with a title and an index. 

The instructon 113 "Filing to Filing Server, Title, In- 
dex Extraction" which the user sent to the scanner 111 
that serves as the information processing apparatus 
30 41 1 , is transmitted from the operation unit 41 to the task 
reception unit 43. The contents are analyzed and plan- 
ning is performed on the results. Then, the execution 
unit 47 executes the final data as information storage 
task PUSH for the filing server 1 1 2, which serves as the 
35 information processing apparatus 412 connected via 
the network. 

The filing server 112, which senses as the informa- 
tion processing apparatus 41 2 connected via the net- 
work, receives the infomnation storage task PUSH from 
40 the reception unit 43, analyzes the received instruction, 
and performs planning on the data. The execution unit 
47 then files corresponding information 115 "contract 
doc-. 

A specific method for communication between the 
45 information processing apparatuses 411 and 412 will be 
described in detail for the forty-ninth embodiment and 
the following. 

[Forty-eighth Embodiment] 

so 

Fig. 135 is a diagram illustrating an example input/ 
output device that can be stored in another apparatus. 

In the example shown in Fig. 135, a user sends an 
instruction 123 to a printer 121 to transmit information 
55 1 25, which is stored in a filing server 122 located at a 
physically remote area, to John 124 who is located at a 
physically farther area. 

The instruction 123 "Action: SEND, What: abc.doc. 
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From: Filing Server, To: John" whicin the user sent to the 
printer 121 that serves as the information processing ap- 
paratus 411, is transmitted from the operation unit 41 to 
the task reception unit 43. The contents are analyzed 
and planning is performed on the results. Then, the ex- 
ecution unit 47 executes the final data as an Instruction 
task for the filing server 122, which serves as the Infor- 
mation processing apparatus 412 connected via the net- 
work. 

The filing server 122, which serves as the informa- 
tion processing apparatus 412 connected via the net- 
work, receives the instruction task from the reception 
unit 43, analyzes the received instruction, and performs 
planning on the data. The execution unit 47 then trans- 
mits corresponding information 125 "abcdoc" to John 
by E-mail. 

A specific method for communication between the 
information processing apparatuses 411 and 41 2 will be 
described in detail for the forty-ninth embodiment and 
the following. 

[Forty-ninth Embodiment] 

Figs. 136A and 136B are flowcharts showing the 
processing for <COMMUNICATE> task. 

<COMMUNICATE> task is performed when arbi- 
trary apparatuses A and B need communicate with each 
other. In Figs. 1 36A and 1 368 are shown the processing 
that the apparatus A (a source) that transmits a COM- 
MUNICATE request is to execute for communication 
with the apparatus B (destination) that receives a COM- 
MUNICATE request. 

At step SI 31 2. the apparatus A contacts the appa- 
ratus B to establish the connection for communication. 
At step SI 31 3 a check is performed to detemnine where 
a reply is received from the apparatus B. If there is no 
response from the apparatus B, at step SI 314 a check 
is performed to determine whether or not a user or an- 
other person should be notified of no response. If noti- 
fication is necessary, at step SI 31 5 a notification is is- 
sued to a user by an appropriate means/medium. At 
step SI 31 6, according to the decision/assumption of the 
system or the setup by the user, a check is performed 
to determine whether the task should be re-execute by 
the system later If the re-execution of the task is nec- 
essary, at step SI 31 7 the task Is added to the task table, 
and the processing Is thereafter terminated- If the re- 
execution of the task is not required, at step SI 318 the 
task is deleted, and is added to "failed list". The process- 
ing is thereafter terminated. 

When, at step S1313, a response is received, at 
step SI 31 9 whether the response is "Will call back" is 
determined. If the response is "Will call back" the appa- 
ratus B will call back the apparatus A later At step 
SI 320, a check is performed to determine whether or 
not the user or another person should be notified of the 
response. If notification is necessary, at step SI 321 the 
notification is issued to the user by an appropriate 



means/medium. At step SI 322 the system adds the task 
to the task table, and remembers that the apparatus B 
will call back so as to cope with the call back. The 
processing is thereafter terminated. 

5 If, at step SI 31 9, the response is not "Will call back" 

at step SI 323 whether the response is "Contact uter- 
is determined. If the response is "Contact later" it is as- 
sumed that the apparatus B wants a contact later, and 
the above described processes at step 81314 and the 

10 following steps are performed. When, at step SI 323, the 
response is not "Contact later" it is assumed that the 
response is "OK" and at step S1 324 the response is con- 
firmed. If the response is not "OK" an error occurs. At 
step S1331 an error message is displayed. 

15 When the response Is "OK" at step S1325 a check 
is performed to determine whether or not a negotiation 
with the apparatus B is required for the current opera- 
tion. If the negotiation is required, at step SI 326 a ne- 
gotiation is initiated. The object to be negotiated in- 

20 dudes a data formal for data execute and protocol/ver- 
sion exchange, and a data format for other information 
required in advance for correct communication. This ne- 
gotiation is repeated until the communication base used 
in common for the apparatuses A and B is established. 

2S At step 81327 information to be transmitted is is- 

sued as an request from the apparatus A to the appa- 
ratus B. At step S1328, whether or not a response is 
received. When the response is received, at step SI 329 
a check is performed to determine whether there is an 

30 action to be executed relative to the received response. 
The action is, for example, conversion of a format or 
erasure of an unnecessary header The object of the 
communication includes actions such as printing, trans- 
mission, filing, voice output and display. This means that 

35 necessary actions should be executed, and at step 
SI 330 these actions are added to the task table. 

When, at step SI 328, there is no response, the 
above described processes at step 81 314 and the fol- 
lowing steps are performed. The above processing loop 

40 is continued until requested information is obtained or 
until the system can not continue the processing. 

[Fiftieth Embodiment] 

45 Figs. 137A and 137B are flowcharts showing the 
processing for <PULL> task. 

<PULL> task is performed when an arbitrary appa- 
ratus A needs to extract information from an apparatus 
B. In Figs. 137A and 137B are shown the processing 

50 that the apparatus A (a source) that transmits a PULL 
request is to execute for communication with the appa- 
ratus B (destination) that receives a PULL request. 

At step SI 41 2, the apparatus A contacts the appa- 
ratus B to establish the connection for communication. 

55 At Step SI 41 3 a check is performed to determine where 
a reply is received from the apparatus B. If there is no 
response from the apparatus B, at step SI 41 4 a check 
is performed to determine whether or not a user or an- 
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other person should be notified of no response. If noti- 
fication is necessary, at step S1415 a notification is is- 
sued to a user by an appropriate nneans/medium. At 
step SI 41 6, according to the decision/assumption ot the 
system or the setup by the user, a check is performed 
to determine whether the task should be re-execute by 
the system later If the re-execution of the task is nec- 
essary, at step S1 41 7 the task is added to the task table, 
and the processing is thereafter terminated- If the re- 
execution of the task is not required, at step S1418 the 
task is deleted, and is added to "failed list". The process- 
ing is thereafter terminated. 

When, at step SI 41 3. a response is received, at 
step 81419 whether the response is "Will call back" is 
determined. If the response is "Will call back" the appa- 
ratus B will call back the apparatus A later At step 
81420, a check is performed to determine whether or 
not the user or another person should be notified of the 
response. W notification is necessary, at step 81 421 the 
notification is issued to the user by an appropriate 
means/medium. At step 81 422 the system adds the task 
to the task table, and remembers that the apparatus B 
will call back so as to cope with the call back. The 
processing is thereafter terminated. 

If, at step 81 41 9, the response is not "Will call back" 
at step 81 423 whether the response is "Contact later" 
is determined. If the response is "Contact later" it is as- 
sumed that the apparatus B wants a contact later, and 
the above described processes at step 81414 and the 
following steps are performed. When, at step 81423. the 
response is not "Contact later" it is assumed that the 
response is "OK" and at step 81 424 the response is con- 
firmed. If the response is not "OK" an error occurs. At 
step 81431 an error message is displayed. 

When the response is "OK" at step 81425 a check 
is performed to determine whether or not a negotiation 
with the apparatus B is required for the current opera- 
tion. If the negotiation is required, at step 81426 a ne- 
gotiation is initiated. The object to be negotiated in- 
cludes a data format for data execute and protocol/ver- 
sion exchange, and a data format for other information 
required in advance for correct communication. This ne- 
gotiation is repeated until the communication base used 
in common for the apparatuses A and B is established. 

At step 81427 information to be extracted is issued 
as a request from the apparatus A to the apparatus B. 
At step 81428, whether or not requested infomnation is 
received. When the requested information is received, 
at step 51429 a check is performed to determine wheth- 
er there is an action to be executed relative to the re- 
ceived response. The action is, for example, conversion 
of a format or erasure of an unnecessary header The 
object of the extraction of information includes actions 
such as printing, transmission, filing, voice output and 
display. This means that necessary actions should be 
executed, and at step 81430 these actions are added 
to the task table. 

When, at step SI 428, requested information is not 



acquired, the above described processes at step SI 41 4 
and the following steps are performed. The above 
processing loop is continued until requested information 
is obtained or until the system can not continue the 
5 processing. 

[Fifty-first Embodiment] 

Figs. 1 38A and 1 388 are flowcharts showing the 
10 processing for <PU8H> task. 

<PU8H> task is performed when an arbitrary appa- 
ratus A need to transmit information to an apparatus B. 
In Figs. 1 38A and 138B are shown the processing that 
the apparatus A (a source) that transmits a PUSH re- 
15 quest is to execute for communication with the appara- 
tus B (destination) that receives a PUSH request. 

At step 81 51 2, the apparatus A contacts the appa- 
ratus B to establish the connection for communication. 
At step 81513 a check is performed to determine where 
20 a reply is received from the apparatus B. if there is no 
response from the apparatus B, at step 81514 a check 
is performed to determine whether or not a user or an- 
other person should be notified of no response. If noti- 
fication is necessary, at step 81515 a notification is is- 
25 sued to a user by an appropriate means/medium. At 
step 81516, according to the decision/assumption of the 
system or the setup by the user, a check is performed 
to determine whether the task should be re-execute by 
the system later If the re-execution of the task is nec- 
30 essary, at step 81517 the task is added to the task table, 
and the processing is thereafter terminated. If the re- 
execution of the task is not required, at step 81518 the 
task is deleted, and is added to "failed list". The process- 
ing is thereafter terminated. 
35 When, at step 81513, a response is received, at 
step 81519 whether the response is "Will call back" is 
determined. If the response is "Will call back" the appa- 
ratus B will call back the apparatus A later At step 
81520, a check is performed to determine whether or 
40 not the user or another person should be notified of the 
response. If notification is necessary, at step 81521 the 
notification is issued to the user by an appropriate 
means/medium. At step 81 522 the system adds the task 
to the task table, and remembers that the apparatus B 
45 will call back so as to cope with the call back. The 
processing is thereafter terminated. 

If, at step 81 51 9, the response is not "Will call back' 
at step 51 523 whether the response is "Contact later" is 
determined. If the response is "Contact later" it Is as- 
50 sumed that the apparatus B wants a contact later, and 
the above described processes at step 81514 and the 
following steps are performed! When, at step SI 523, the 
response is not "Contact later" it is assumed that the 
response is "OK" and at step 81 524 the response is con- 
55 firmed. It the response is not "OK" an error occurs. At 
step S1531 an error message is displayed. 

When the response is "OK" at step 81 525 a check 
is performed to determine whether or not a negotiation 
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with the apparatus B is required for the current opera- 
tion. If the negotiation Is required, at step S1526 a ne- 
gotiation is initiated. The object to be negotiated in- 
cludes a data format for data execute and protocolA/er- 
sion exchange, and a data format for other information 
required in advance lor correct communication. This ne- 
gotiation is repeated until the communication base used 
in common for the apparatuses A and B is established. 

At step S1527 information Jo be transmitted is is- 
sued as a request from the apparatus A to the apparatus 
B. At step SI 528, whether or not a response is received. 
When the response is received, at step SI 529 a check 
is performed to determine whether there is an action to 
be executed relative to the received response. The ac- 
tion is, for example conversion of a format or erasure 
of an unnecessary header The object of the communi- 
cation includes actions such as printing, transmission, 
filing, voice output and display This means that neces- 
sary actions should be executed and at step 81530 
these actions are added to the? task (able. 

When, at step 31528 itieie ii> no response, the 
above described processes at s:ep S1514 and the fol- 
lowing steps are performed The above processing loop 
is continued until requested information is obtained or 
until the system can not continue the processing. 

[Fifty-second Embodiment) 

Figs. 139A and 139B are flowcharts showing the 
processing for <INDICATE> task. 

<INDICATE> task is performed when an arbitrary 
apparatus A needs to send an instruction to an appara- 
tus B. In Figs. 1 39A and 1 39B are shown the processing 
that the apparatus A (a source) that transmits an INDI- 
CATE request Is to execute for communication with the 
apparatus B (destination) that receives an INDICATE re- 
quest. 

At step S161 2, the apparatus A contacts the appa- 
ratus B to establish the connection for communication. 
At step SI 61 3 a check is performed to detenmine where 
a reply is received from the apparatus B. If there Is no 
response from the apparatus B, at step SI 614 a check 
is performed to determine whether or not a user or an- 
other person should be notified of no response. If noti- 
fication is necessary, at step SI 61 5 a notification is is- 
sued to a user by an appropriate means/medium. At 
step S 161 6, according to the dec is ion/assumption of the 
system or the setup by the user, a check is performed 
to determine whether the task should be re-execute by 
the system later If the re -execution of the task is nec- 
essary, at step S161 7 the task is added to the task table, 
and the processing is thereafter terminated. If the re- 
execution of the task is not required, at step SI 61 8 the 
task is deleted, and is added to "failed list". The process- 
ing is thereafter terminated. 

When, at step SI 613, a response is received, at 
step S1619 whether the response is "Will call back" is 
determined. If the response is "Will call back" the appa- 



ratus B will call back the apparatus A later. At step 
SI 620, a check is performed to determine whether or 
not the user or another person should be notified of the 
response. If notification is necessary, at step S1621 the 
5 notification is issued to the user by an appropriate 
means/medium. At step S 1 622 the system adds the task 
to the task table, and remembers that the apparatus B 
will call back so as to cope with the call back. The 
processing is thereafter terminated. 

If, at step S1 61 9, the response is not "Will call back" 
at step S1623 whether the response is "Contact later" 
is determined. If the response is "Contact later" it is as- 
sumed that the apparatus B wants a contact later, and 
the above described processes at step SI 61 4 and the 
following steps are performed. When, at step 51 623, the 
response is not "Contact later" it is assumed that the 
response is "OK" and at step S 1 624 the response is con- 
firmed. If the response is not "OK" an error occurs. At 
step S1631 an error message is displayed. 

When the response is "OK" at step SI 625 a check 
is performed to determine whether or not a negotiation 
with the apparatus B is required for the current opera- 
tion. If the negotiation is required, at step SI 626 a ne- 
gotiation Is initiated. The object to be negotiated in- 
cludes a data format for data execute and protocolA^er- 
slon exchange, and a data format for other infomnation 
required in advance for correct communication. This ne- 
gotiation Is repeated until the communication base used 
in common for the apparatuses A and B is established. 

At step SI 627 according to the operation instruction 
by the user employing the operation unit 41 , an instruc- 
tion Is issued as an request from the apparatus A to the 
apparatus B. At step SI 628, a check is performed to 
determine whether or not a response is received. When 
the response is received, at step SI 629 a check is per- 
formed to determine whether there Is an action to be 
executed relative to the received response. The action 
is, for example, conversion of a format or erasure of an 
unnecessary header. The object of the communication 
includes actions such as printing, transmission, filing, 
voice output and display. This means that necessary ac- 
tions should be executed, and at step SI 630 these ac- 
tions are added to the task table. 

When, at step SI 628, there is no response, the 
above described processes at step SI 61 4 and the fol- 
lowing steps are performed. The above processing loop 
is continued until requested information is obtained or 
until the system can not continue the processing. 

[Fifty-third Embodiment] 

Figs. 140A and 140B are flowcharts showing the 
processing for <RESPOND> task. 

<RESROND> task is performed when an apparatus 
B (destination) that receives a COI^MUNICATE request 
sends a response. In Figs. T40A and 140B are shown 
the processing that Is to be executed by the apparatus 
Bthat receives a COMMUNICATE request (e.g., PULL) 
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from a transmission apparatus (source) A. 

At step SI 71 2 a check is performed to determine 
whether or not a request type is contact. It the request 
is contact, the apparatus A attempts to establish con- 
nection with the apparatus B tor communication. At step 
S1713 the current status or a specific user's set value 
is allocated for status SI. At step SI 31 4 whether the 
status SI is a "Very busy" mode or a "Standby" mode is 
determined. If the status 81 is either mode, it is assumed 
that the apparatus B can not accept the request from 
the apparatus A. Then, at step S1715 the optimal re- 
sponse is decided. A check is performed to determine 
whether or not the apparatus B can call back the appa- 
ratus A any time. If the apparatus B can call back, pro- 
gram control moves to step SI 71 6 and response "Will 
call back" is transmitted. It the apparatus B can not call 
back, at step Si 71 7 response "Contact later" is trans- 
mitted. This means that the apparatus A must contact 
the apparatus B later When, at step S1714, the status 
A1 is neither a "Very busy" mode nor a "Standby" mode, 
response "OK" is transmitted. The processing is there- 
after terminated. 

When the request type is not contact, at step SI 71 9 
a check is performed to determine whether the request 
type is a request for information. If so, at step SI 720 a 
response is transmitted relative to the request tor infor- 
mation, which will be described later in Fig. 141. When 
the request type is not a request for information, at step 
SI 721 a check is performed to determine whether the 
request type is a request for storage of information. If 
so, at step SI 722 a response relative to the request for 
storage of information, which will be described later in 
Fig. 142, is issued. If the request type is not a request 
for storage of information, at step SI 723 a check is per- 
formed to determine whether the request type is a re- 
quest for an instruction. If so, at step S1724 a response 
relative to the request for an instruction, which will be 
described later, is issued. If the request type is not a 
request for an instruction, at step S1725 an optimal re- 
sponse is issued, and the processing is thereafter ter- 
minated. 

Fig. 1 41 is a flowchart showing a response task rel- 
ative to the request for information. In this flowchart, a 
process will be explained that is initiated when the re- 
quest type is a request for information (step Si 720 in 
Figs. 140 A and 1408). 

At step SI 812 a check is performed to detemriine 
whether or not negotiation is required with the apparatus 
A that issued the request for an instruction. If the nego- 
tiation is required, program control moves to step 
S1B13, whereat the negotiation is initiated. The object 
to be negotiated includes a data format for data execute 
and protocolA/ersion exchange, and a data format for 
other information required in advance for correct com- 
munication. This negotiation is repeated until the com- 
munication base used in common for the apparatuses 
A and B is established. 

Following this, at step S1814 fetching of requested 



information is performed. At step SI 81 5, a check is per- 
formed to determine whether or not the requested infor- 
mation is found. If the information is not found, at step 
SI 81 6 response = NG is transmitted and the processing 
5 is terminated. 

If the requested information is found, at step SI 81 7 
a check is performed to determine whether the informa- 
tion should be transferred immediately. If the immediate 
transmission is required, program control moves to step 
10 SI 820, whereat a check is performed to determine 
whether or not data formatting or data conversion is re- 
quired. If data formatting or data conversion is required, 
at step S1819 it is performed. Then, at step S1820 the 
requested information is transmitted as a response and 
15 the processing is thereafter terminated. If the immediate 
transmission is not required, program control moves to 
step S1821 , whereatataskfortransmittingthe data later 
is added to the task table. At step S1 822 response = OK 
is issued, and the processing is terminated. 
20 Fig. 1 42 is a flowchart showing a response task rel- 
ative to the request for storage of information. In this 
flowchart, a process will be explained that is initiated 
when the request type is a request for storage of infor- 
mation (step SI 722 in Figs. 140A and 140B)- 
25 At step SI 912 a check is performed to determine 
whether or not negotiation is required with the apparatus 
A that issued the request for an instruction. If the nego- 
tiation is required, program control moves to step 
S2013, whereat the negotiation is initiated. The object 
30 to be negotiated includes a data format for data execute 
and protocol/version exchange, and a data format for 
other information required in advance for correct com- 
munication. This negotiation is repeated until the com- 
munication base used in common for the apparatuses 
35 A and B is established. 

Following this, at step SI 91 4 infornnation transmit- 
ted for the apparatus A is acquired. At step SI 915 
whether or not the information is received is determined. 
If the information is not yet received, program control 
40 goes to step SI 916, whereat response = NG is returned, 
and the processing is terminated. If the information re- 
ceived, whereat a check is performed to determine 
whether or not data formatting or data conversion is re- 
quired. If data formatting or data conversion is required, 
45 at step SI 91 8 it is performed. Then, at step SI 91 9 re- 
sponse = OK is issued, and the processing is terminat- 
ed. 

Fig. 143 is a flowchart showing a response task rel- 
ative to a request for an instruction. In this flowchart, a 

50 process will be explained that is initiated when the re- 
quest type is a request for an instruction (step SI 724 in 
Figs. 140A and 140B). 

At step S2012 a check is performed to determine 
whether or not negotiation is required with the apparatus 

55 A that issued the request for an instruction. If the nego- 
tiation is required, program control moves to step 
SI 91 3. whereat the negotiation is initiated. The object 
to be negotiated includes a data format, conversion pro- 
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tocol, upgrading of a version, adjustnnents associated 
with alteration of information types, and specification of 
the instruction contents. The negotiation process is 
planed before this process is initiated. 

FoMowing this, at step S201 4 instruction infornnation s 
is interpreted to decide a process to be performed next. 
At step S201 5 a check is performed to determine wheth- 
er or not the process can be executed. If the process 
can not be performed, response = NG is returned and 
the processing is terminated. to 

When the process can be performed, program con- 
trol moves to step S2017, whereat whether or not im- 
mediate execution is required. If the process need not 
be performed anmediately, program control goes to step 
S2020 whereat a task for designating the performance 
of a corresponding process is added to the task table, 
and response - OK is returned. The processing is there- 
after tcrminalod 

If the process should be immediately performed, 
progiarrs control advances lo step S2018, whereat the 
instfucted piocess is performed. The processing is 
thereallef terminated 

(Fifty fourth Embodiment] 

25 

An explanation of PULL task, for example pull print- 
ing tor extracting and prrnting information, i.e., an exam- 
ple print on demand process wilt now be given, by re- 
ferring to Fig 1 33 

In the example in Fig 1 33, a user sends an instruc- 30 
tion 1333 to a printer A1331 to extract information 1334 
•contract doc" included in an apparatus B1332 that is 
located physically away from the printer A and to permit 
the printer A to print the information 1334. The appara- 
tus B can be a printer, a scanner, a desktop PC, or a 35 
filing server 

The contents "Please fetch 'contract.doc' from B 
and print it" which is an instruction from the user to the 
printer A1 331 . may be a command input using a mouse, 
or via a keyboard, by voice or a digitizer, or may be a 40 
sentence in the natural language by character recogni- 
tion. 

When the apparatus B is a different device type from 
the apparatus A, e.g., a scanner, the information format- 
ting is TIFF BMP or JPEG, and format conversion is re- 45 
quired. This data conversion may be performed either 
by the apparatus B or by the printer A to obtain the ap- 
propriate data formal. When the apparatus B is a printer 
and information of the apparatus B has a format that the 
printer A can understand (e.g., when both apparatuses 
A and B are PCL printers), data conversion is not re- 
quired. In other cases, even when both apparatuses are 
printers, data conversion is necessary. It should be not- 
ed that the apparatus A is not necessary a printer, and 
may be another output device, such as a facsimile ma- 
chine, a copier or a display. 

The transmission of infomnation for the PULL task 
will now be described Fig. 144 is a diagram showing an 



example pull session. In this example, the apparatuses 
A and B are printers, but as is described above, the ap- 
paratuses A and B need not necessarily be the same 
device type, and can be a scanner, a printer, a copier or 
a facsimile machine. 

The apparatus A performs the procedures in Figs. 
1 37A and 1 37B while the apparatus B performs the pro- 
cedures in Fig. 141 . First, the apparatus A contacts the 
apparatus B to establish a connection for communica- 
tion (step S1412). When the response is OK (steps 

51 424 and SI 822), a request for information (informa- 
tion <abc> in this example) and an information format, 
if necessary, are transmitted to the apparatus B (step 
SI 427). 

The apparatus B fetches the requested information 
(step 51814), converts the format, as needed (step 
SI 81 9), and transmits information in response to the re- 
quest (step SI 820). 

When the apparatus A extracts information from the 
apparatus B, it executes a necessary action in accord- 
ance with the instruction from the user or the determi- 
nation of the system. The necessary action is, for exam- 
ple, printing, and in this case^ pull printing is performed, 
i.e., information is extracted from a remote area and 
printed. In other words, print on demand can be accom- 
plished. 

Another necessary action is a display. In this case, 
pull viewing is performed^ i.e., information is extracted 
from a remote area and displayed. In other words, view 
on demand can be accomplished. An additional neces- 
sary action is audio output. In this case, pull hearing is 
performed, i.e., information is extracted trorri a remote 
area and can be listened to. In other words, hear on de- 
mand can be accomplished. 

Similarly, further necessary actions are storage, fil- 
ing and transferring. Any type of information can be ex- 
tracted; a document, a file, electronic mail, a schedule, 
a status report, a log, voice mail or telephone call. 

[Fifty-fifth Embodiment] 

Fig. 1 45 is a diagram showing another example pull 
session. Along the same procedure as in Fig. 144, the 
apparatus A requests information <abc> to the appara- 
tus B. However, the apparatus B does not immediately 
transmits the requested information (No at step SI 81 7). 
A task for transmitting the data later is added to the task 
table (step SI 821), and response = OK is returned to 
terminate the processing (step SI 822). In this case, the 
transfer of the requested information is a print request 
from the apparatus B after a new session is established. 

[Fifty-sixth Embodiment] 

Fig. 146 is a diagram showing an additional exam- 
ple pull session. In this example, a negotiation is held 
before the transfer of necessary information (steps 

51425 and SI 426 and steps SI 81 2 to SI 822). Since 
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the information to be extracted by the apparatus A is a 
file, the apparatus A requests the apparatus B for a list 
of files. The apparatus B transmits the list of files to the 
apparatus A, From the list, the apparatus decides infor- 
mation necessary for the system or instructs the input 
of information necessary for a user. The apparatus A 
then decides the information to be requested, and is- 
sues a request for information to the apparatus B in the 
same manner as described above. 

[Fifty-seventh Embodiment] 

Fig. 147 is a diagram showing an example of Hear 
On Demand. In this example, "Read out New Mail" is 
input. While, in Fig. 147, an input window is employed 
for this input, any input device, such as a voice input 
device, a keyboard or a digitizer, can be employed as in 
the previous examples. 

In this example, the above command is transmitted 
to an apparatus A (printer) at Kosugi, and mail to be out- 
put is stored in an apparatus B (desktop PC) at Shi- 
momaruko. The apparatuses A and B may be located 
in the same place, or in the same or different network 
domain. This session will be described while referring 
to Fig. 148. 

Fig. 1 48 is a diagram showing a further example pull 
session. Information type is electronic mail and a re- 
quested format is voice. The procedures are, however, 
the same as previously described. 

The apparatus A requests the information (mail) 
(step SI 427), the apparatus B converts the requested 
data into a suitable format (voice format in this example) 
(step S1B19), and sends a response. Then, the appa- 
ratus A performs an optimal action. Since the optimal 
action is output of information by voice, the contents of 
mail is read aloud. While, in this example, the apparatus 
B converts text data into voice format, when the appa- 
ratus B can not convert the data the apparatus A may 
perform such conversion before the data output. 

[Fifty-eighth Embodiment] 

Fig. 149 is a diagram for explaining another pull 
task. In this example, an apparatus C is employed to 
issue a command to an apparatus A. The apparatus A 
is a display, and an instruction "Display my schedule" is 
issued to the apparatus C. Although the communication 
is not specified in the instruction, it means that a user 
extracts schedule data from the apparatus B (user's 
desktop PC) and displays it on the apparatus A. 

While the information type is schedule information, 
the procedures for this is the same as explained previ- 
ously. 

[Fifty-ninth Embodiment] 

Fig. 150 is a diagram for explaining an example 
push task. In this example, an apparatus A that receives 



a command is a scanner. The command shown in Fig. 
150 is issued to instruct the apparatus A to transmit a 
document obtained by scanning to an apparatus B 
(printer) and to instruct the apparatus B to hold the doc- 
5 ument instead of printing it. This session will now be de- 
scribed. 

Fig. 151 is a diagram showing a session for the push 
task in Fig. 150. As is described above, the apparatus 
A is a scanner and the apparatus B is a printer. 

10 The apparatus A perfomns the procedures in Figs. 
1 38A and 1 38B while the apparatus B performs the pro- 
cedures in Fig. 1 32. First, the apparatus A contacts the 
apparatus 8 to establish a connection for communica- 
tion (step S1512). Assume that the response is OK 

IS (steps SI 524 and SI 71 8). 

Then, transfer of information (information <abc>) is 
requested. Since the request type is a request storage 
of information, the apparatus performs the procedures 
in Fig. 142. At the negotiation (step S1526 and SI 91 3), 

20 the apparatus B requests the apparatus A for a fornnat 
(<xyz>) of information to be transferred. 

The apparatus A changes the format, if necessary, 
and transmits the requested information (step SI 527). 
Upon the receipt of the information, the apparatus B per- 

25 forms a required action according to the decision of the 
transmission side, the user or the apparatus B. The ac- 
tion is, for example, format conversion, filing, index 
preparation, data extraction, or title extraction. The nec- 
essary action in this embodiment is to hold a received 

30 document instead of printing it. 

[Sixtieth Embodiment] 

Fig. 152 is a diagram showing another example 
35 push task. The apparatus A, which is a scanner in this 
example, sends a request to the apparatus B (scanner) 
and the apparatus C (filing server) to push information, 
to file a document on a reception side and to extract its 
title and index, 

40 The specific processing is the same as that in the 
fifty-ninth embodiment. The session between the appa- 
ratuses A and B will now be described while referring to 
Fig. 153. The same thing is applied for the session be- 
tween the apparatuses A and C. Communication be- 

45 tween the apparatuses A and B and communication be- 
tween the apparatuses A and G are performed in paral- 
lel. 

Fig. 153 is a diagram showing an example push 
session. The apparatuses A and B are scanners, when 

50 the apparatus B receives data, it performs a required 
action. In this case, the apparatus A transmits infonma- 
tion for designating an associated action that should be 
executed. The action is filing and extraction of atrtle and 
an index. While in this example the action is instructed 

55 from the apparatus A, the apparatus B may decide an 
action, and may abandon a received action. 
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[Sixty-first Embodiment] 

Fig. 154 is a diagram showing another example 
push task, which is a transmission of electronic mail. An 
apparatus A (printer) at Kosugi is to transmit electronic 
mail "Get design from Bill" to an apparatus B (having a 
different network sub-domain from John's PC and the 
apparatus A) in New York. 

Any method can be used for mail transmission. 
First, when the apparatus A has a mail transmission 
function, it transmits the mail. If the apparatus A does 
not have such a function, it transmits mail information to 
another apparatus on the network, e.g., a mail server, 
and asks the mail transmission. 

[Sixty-second Embodiment] 

The instruction task INDICATE will be described 
while referring to Fig. 132. A user sends an instruction 
93 to a copier 91 that is physically located in a remote 
area in order to permit a MyDesk 92, a desktop personal 
computer, to display the current location of the user. 

The instruction 93 "Display present place on 
<MyDesk>", which is sent to the copier 91 that serves 
as the information processing apparatus 411, is trans- 
mitted from the operation unit 41 to the task reception 
unit 43. The contents are analyzed and planning is per- 
formed on the results. Then, the execution unit 47 exe- 
cutes the final data as an instruction task INDICATE for 
a PC 92, which serves as the information processing 
apparatus 412 connected via the network. 

The PC 92, which is the information processing ap- 
paratus 412 connected via the network, receives in- 
structed information from the reception unit 43, analyzes 
a received instruction, and perform planning on the re- 
sults. Then, the execution unit 47 displays the location, 
such as "Now in conference Shimomaruko A111 confer- 
ence room" on a screen. 

Transmission of information for each session be- 
tween the information processing apparatuses 411 and 
412 will be explained while referring to Fig. 155. Fig. 155 
is a diagram showing transmission of information for 
each session between a copier, which serves as the in- 
formation processing apparatus 411 for instruction task 
INDICATE, and a PC. which serves as the information 
processing apparatus 41 2. 

The transmission side performs the processing in 
Figs. 139Aand 139B, while the reception side performs 
the processing in Fig. 1 43. The communication contents 
are an instruction and information for processing corre- 
sponding to the instruction. The transmission of infor- 
mation is the same as that in the above described push 
task. 

In response to the instruction request from the ap- 
paratus A, the apparatus B transmits <xyz> instruction 
type that can be understood by the apparatus B Upon 
receipt of the information, the apparatus A changes the 
instruction information, as needed. 



Then, the apparatus A issues, to the apparatus B, 
an instruction and necessary information to display on 
the PC screen the fact that a conference is being held 
and the current location. The apparatus B receives the 
s information, performs the process in consonance with 
the information, and displays the location of the user on 
the screen, as is instructed. 

[Sixty-third Embodiment] 

10 

Another example instruction task INDICATE will 
now be described while referring to Fig. 130. A user 
sends an instruction 71 to a copier 74 at a physically 
remote area in order to set a facsimile machine 75 at 
75 Shinkawasaki so that it transfers, to a printer 77 of Mr 
Toyoda at Kosugi, a document that is received from Mr. 
Suzuki. 

The instruction 71 "Please advice My Fax to transfer 
any document sent from Mr. Suzuki to Mr. Toyoda' which 

^0 is sent to the copier 74 that serves as the Information 
processing apparatus 411 , is transmitted from the oper- 
ation unit 41 to the task reception unit 43. The contents 
are analyzed and planning is performed on the results. 
Then, the execution unit 47 executes the final data as 

2S an instruction task INDICATE for the facsimile machine 
75, which serves as the information processing appara- 
tus 412 connected via the network. 

The facsimile machine 75, which is the information 
processing apparatus 412 connected via the network, 

30 receives instruction information from the reception unit 
43, analyzes a received instruction, and perform plan- 
ning on the results. Then, the execution unit 47 alters 
the condition setting. The transmission of information for 
each session between the information processing ap- 

55 paratuses 411 and 412 will be described while referring 
to Fig. 156. 

Fig. 1 56 is a diagram showing transmission of infor- 
mation for each session between a copier, which serves 
as the information processing apparatus 411 in this ex- 
ample instruction task INDICATE, and a facsimile ma- 
chine, which swerves as the information processing ap- 
paratus 412. The transmission side performs the 
processing in Figs. 139A and 1 39B, while the reception 
side performs the processing in Fig. 143. The transmis- 

^5 sion of information is the same as that in the above de- 
scribed example. 

In response to the instruction request from the ap- 
paratus A, the apparatus B transmits <xyz> instruction 
type that can be understood by the apparatus B. Upon 

so receipt of the information, the apparatus A changes the 
instruction information, as needed. In this example, the 
apparatus A transmits to the apparatus B an instruction 
and necessary information to transfer, to Mr. Toyoda, a 
document that is transmitted from the Mr Suzuki and 

55 addresses the user Upon receipt of the information, the 
apparatus B performs the required processing in con- 
sonance with the information, and sets a designated 
condition. 
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[Sixty-fourth Embodiment] 

An additional example instruction task INDICATE 
will now be described while referring to Fig. 1 35. A user 
sends an instruction l23toascanner121 at a physically 
remote area in order to instruct a filing server 122 to 
transfer a stored document 125 •abc.doc" to John 124. 

The instruction 123 "Action: SEND, What: r, From: 
Filing Server, To: John" which is sent to the scanner 1 21 
that serves as the information processing apparatus 
41 1 , is transmitted from the operation unit 41 to the task 
reception unit 43. The contents are analyzed and plan- 
ning is performed on the results. Then, the execution 
unit 47 executes the final data as an instruction task IN- 
DICATE for the filing server 122, which serves as the 
information processing apparatus 412 connected via 
the network. 

The filing server 122, which is the information 
processing apparatus 412 connected via the network, 
receives instruction information from the reception unit 
43, analyzes a received instruction: and perform plan- 
ning on the results. Then, the execution unit 47 transmits 
a document. The transmission of information for each 
session between the information processing apparatus- 
es 411 and 412 will be described while referring to Fig. 
157, 

Fig. 157 is a diagram showing transmission of infor- 
mation for each session between a scanner, which 
serves as the information processing apparatus 411, 
and a filing server, which serves as the information 
processing apparatus 412. The transmission side per- 
forms the processing in Figs. 1 39A and 1 39B, while the 
reception side performs the processing in Fig, 143. The 
transmission of information is the same as that in the 
above described example. 

In response to the instruction request from the ap- 
paratus A, the apparatus B transmits <xyz> instruction 
type that can be understood by the apparatus B. Upon 
receipt of the information, the apparatus A changes the 
instruction information, as needed. In this example, the 
apparatus A transmits to the apparatus B an instruction 
and necessary information to transfer, to John, a docu- 
ment "abc.doc" that is stored in the filing server 

Upon receipt of the information, the apparatus B 
performs the required processing in consonance with 
the information, decides an optimal transmission meth- 
od, such as E-mail, and transmits the document to John 
as is instructed. 

[Sixty-fifth Embodiment] 

Fig. 158 is a diagram showing an operation along 
the procedures of another apparatus. In the sixty-fourth 
embodiment explained while referring to Fig. 1 35. a user 
sends an instruction to the scanner 1 21 that is located 
at a physically remote area, in order to instruct the filing 
server 122 to transfer the stored document 125 "abc. 
doc" to the John's desktop 124. And the scanner 121 



interprets contents and executes the instruction task I N - 
DICATE. 

In this embodiment, however, a printer 351 receives 
operation information 353 from a filing server 352 and 
5 displays it, and a user performs operatbn in accordance 
with the display and transmits the operation data to the 
filing server 352. In this manner, the same effect as in 
the sixty-fourth embodiment can be obtained. 

When a user requests the printer 351 , which serves 
10 as the information processing apparatus 41T to operate 
the filing server 352, a planning decision unit 46 in the 
printer 351 makes a plan that necessary operation data 
353 should be obtained to implement the operation pro- 
cedures of the filing server 352. The process execution 
15 unit 47 executes the information acquisition task PULL 
for the filing server 352, which is the information 
processing apparatus 412 connected via the network. 

The filing server 352, which is the information 
processing apparatus 412 connected via the network, 
20 receives instruction request information from the recep- 
tion unit 43, analyzes a received request, and perform 
planning on the results. Then, the execution unit 47 
transmits the operation data 353. 

A screen is provided on the operation unit 41 of the 
25 scanner 351. which receives the operation data 353 
from the filing server 352, and the operation unit 41 dis- 
plays, on the screen, the procedures of the filing server 
352 that correspond to the operation data 353. 

According to the displayed operation screen, the in- 
30 formation concerning the operation of the user is trans- 
mitted to the filing server 352 in the same manner The 
filing server 352 that receives the operation of the user 
interprets it and performs a necessary process. As a re- 
sult, transmission of the file "abc.doc" to John, which is 
35 the object to the user, is enabled. 

In the above description, the filing server 352 trans- 
mits the operation data 353 for implementing the oper- 
ation procedures of the filing server 352; however, an 
operation data server may be prepared for managing 
40 the operation data 353. and may transmit the operation 
data 353. The operation data may also be transmitted 
from the third information processing apparatus that re- 
ceives the operation data 353 of the filing sender 352, 
like the printer 351 that sen/es as the information 
45 processing apparatus 411. The printer 351 may be re- 
placed with a scanner 

[Sixty-sixth Embodiment] 

50 Fig. 162 is a diagram showing the relationship be- 
tween the change to three statuses of a print job and 
processes corresponding to the individual statuses. An 
output device (printer), which serves as the information 
processing apparatus 411. receives a print job that in- 

55 dudes a document for which a user instructed printing. 

First, an input job determination unit 391 deter- 
mines whether a received print job should be immedi- 
ately printed or should be held until a print execution 
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condition is established. The result is transmitted to a 
pending job managennent unit 392 or an output wait job 
management unit 394. 

The pending job nnanagement unit 392 stores the 
received print job as a pending job in a pending job table 5 
393. The pending job management unit 392 examines 
the pending job table 393 for the job for which print ex- 
ecution condition is established. If such a job is found, 
it is extracted from the pending job table 393. The pend- 
ing job for which the print condition is established is io 
transmitted to the output wait job management unit 394. 

The output wait job management unit 394 stores a 
received print job as an output wait job in an output wait 
job table 395. When printing at the output unit 396 Is 
enabled, the output wait job is extracted from the output i5 
wait job table 395 and transmitted to the output unit 396. 

The output unit 396 prints the received job. When 
printing is successful, the print job is transmitted to an 
output job management unit 397 to store the print job 
that has been output. 20 

The output job management unit 397 stores, as an 
output job, the received print job that has been output 
in an output job table 398. 

As is described above, the received print job is 
changed into three statues and processed. The print ex- 2S 
ecution condition of a pending job is not only a condition 
where the job is held until the user instructs printing, but 
also a condition where printing is initiated at a designat- 
ed time, or a condition where, if the main printer is other 
than printer A that serves as the information processing 30 
apparatus 411, the main printer is ready for printing. 

Figs. 163A and 163B are flowcharts showing the 
printing processing for this embodiment. The process- 
ing of a print task "PRINT (A, 'abc.doc')" or "PRINT (not 
A, 'abc.doc')" is perfornned when an instruction is issued 35 
to the printer A, which is the information processing ap- 
paratus 411 , to print document "abcdoc" by the printer 
A and by a printer other than the printer A. 

When the print task PRINT is activated, at step 
S4001 a check Is perfornned to determine whether or not 40 
a document to be printed Is included in a print job. When 
there is no document to be printed, program control 
moves to step S4002. At step S4002 a check is per- 
formed to determine whether the document is included 
among the documents in the print jobs that are entered 
in the pending job table 393, the output wait job table 
395 or the output job table 398. When, at step S4003, 
the document is not found, program control advances 
to step S4004. 

At step S4004 the infomnation acquisition task 50 
PULL is performed in order to acquire the document 
from another information processing apparatus 412. If, 
at step S4005, it is ascertained that the document is suc- 
cessfully acquired, program control advances to step 
S4006. If the acquisition of the document fails, program ss 
control moves to step S4016, whereat ah error is dis- 
played and another necessary process is performed: 
The processing Is thereafter terminated. At step S4006 



a check is performed to determine whether the docu- 
ment to be printed can be printed by the printer A, which 
is the information processing apparatus 411, or by a 
printer other than the printer A. If the conversion of the 
document into a printer description language is re- 
quired, program control moves to step S4007. 

At step S4007 a check is perfornned to determine 
whether or not the printer A, which is the information 
processing apparatus 411, can convert the document 
form. If the conversion is possible, program control goes 
to step S4008, whereat the document to be printed is 
converted into a suitable form. When the conversion is 
impossible, program control goes to step 54009, 
whereat instruction INDICATE is issued to another in- 
formation processing apparatus 412 to convert the doc- 
ument to be printed. According to the instruction INDI- 
CATE, the information processing apparatus 412 may 
immediately perform the conversion and return the re- 
sult, or may return only an acknowledgement and per- 
form conversion after printing Is Instructed later. 

At step S4010 a check is performed to determine 
whether or not the conversion result is successful. If it 
is successful, program control advances to step S4011 . 
If the conversion fails, program control moves to step 
S4016, whereat an error is displayed and another re- 
quired process is performed. The processing is thereaf- 
ter terminated. 

At step S4011 a check is performed to determine 
whether or not the printer A, which is the information 
processing apparatus 411, should print a document. 
When it is determined the printer A should print the doc- 
ument, program control advances to step S4012, and 
the document is printed. When the information process- 
ing apparatus 412 should print the document, program 
control goes to step S401 3, and the information storage 
task PUSH and print task PRINT for the information 
processing apparatus 412 are performed 

At step S4014, whether or not the printing is suc- 
cessful is determined. When the printing is successful, 
program control advances to step S4015, and the print 
job Is regarded as the output job, which is then stored 
in the output job table 398. The other necessary process 
is performed, and the processing is terminated. When 
the printing fails, program control moves to step S4015, 
whereat an error is displayed and another required proc- 
ess is performed. The processing is thereafter terminat- 
ed. 

The conversion of the document at step S4008 in- 
cludes a conversion of a normal text document into a 
document in a printer description language; a conver- 
sion of a document in a specific printer language Into 
another printer language that the employed printer can 
understand; a conversion of an image format document 
read by a scanner or a digital camera into a document 
in a printer language; or a conversion of a document in 
the common language in WWW Into a printer language. 

The conversion instruction INDICATE at step 
S4009 may be an instruction to the information process- 
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ing apparatus 41 2 that has a specific application to con- 
vert a docunnent in a language inherent to the corre- 
sponding application into a document in a printer lan- 
guage that can be printed; or an instruction to the infor- 
mation processing apparatus 412 that has a specific s 
printing function to convert a document in a language 
inherent to a specific printer into a document in a printer 
language that can be printed. 

Fig. 164 is a diagram showing the relationship be- 
tween the print job and the processing, which are ex- io 
plained while referring to Figs. 1 62 and 1 63, and the us- 
er's operation. When a user permits the scanner A411, 
which serves as the information processing apparatus 
412. to read document "abc.doc" 144 and issues an in- 
struction "Send *abc.doc' to printer B" the printer B 41 2, ^5 
which serves as the information processing apparatus 
41 1 that receives the print job, does not print -abc.doc" 
immediately, but regards it as a pending job and stores 
it in the pending job table 393. 

When the user sends an instruction "Show pending 20 
documents" the document "abc.doc" is displayed on the 
operation pane! of the printer B. Furthermore, when the 
user designates the document "abc.doc" and issues an 
erasure instruction "Delete pending document" the des- 
ignated document is deleted from the pending job table 2S 
393. 

When the user designates the document "abc.doc" 
and issues an instruction "Print pending document" the 
document "abc.doc" is changed to a print wait job, which 
is then stored in the output wait job table 395. When the 30 
printing is enabled, it is printed (415). 

When the printing is successful, the document "abc. 
doc" is changed to the output job, which is then stored 
in the output job table 395. In addition, when the user 
designates the output document "abc.doc" and Issues 3S 
an erasure instruction "Delete printed document" the 
designated document is deleted from the output job ta- 
ble 398. 

Fig. 165 is a diagram showing an example display 
screen that is used by a user to operate a print job in the 40 
three statues as explained in Fig. 162. In accordance 
with the operation by a user, an instruction from the in- 
formation processing apparatus 412, or the result ob- 
tained by the process performed by the information 
processing apparatus 41 1 , the print jobs in the three sta- ^5 
tuses are operated arbitrarily, and are added as a new 
task to the task table 399. 

For example, a user utilizes the operation screen in 
Fig. 1 65 to delete (4209) a print job stored in the pending 
job table 393 (4201 ), to send (4208) the print jot to print so 
using another printer, or to print It immediately. It is also 
possible to instruct to increase the priority (4210) of a 
print job stored in the output wait job table 395 (4202) 
so that it will be printed first, or a print job, or to print 
(4204) again the print job stored in the output job table 5S 
398 (4203). 

The present invention can be applied either for a 
system that is constructed by a plurality of apparatuses 



(e.g., a host computer, an interface device, a display) or 
for an apparatus including a single device. 

The following is also included within the scope of 
the present invention: in order to operate various devic- 
es to implement functions in the above embodiments, 
software program code for implementing the functions 
in the previous embodiments are supplied to a computer 
in an apparatus or in a system that is connected to var- 
ious devices, and in consonance with the program, the 
computer (or a CPU or an MPU) in the system or the 
apparatus operates the devices to accomplish the func- 
tions in the above embodiments. In this case, the pro- 
gram code read from a memory medium accomplishes 
the functions of the above described embodiments. And 
the program code and means for supplying the program 
code to the computer, e.g., a memory medium on which 
such program code is recorded, constitute the present 
invention. 

A memory medium for supplying such program 
code can be, for example, a floppy disk, a hard disk, an 
optical disk, a magneto optical disk, a CD-ROM, a CD- 
R, a magnetic tape, a nonvolatile memory card, or a 
ROM. 

In addition, not only for a case where the functions 
in the previous embodiments can be performed when 
program code is read and executed by the computer, 
but also for a case where, according to an instruction in 
the program code, an OS (Operating System) running 
on the computer, or another application software pro- 
gram, interacts with the program code to accomplish the 
functions in the above embodiments, this program code 
can be included in the embodiments of the present In- 
vention. 

Furthermore, the present invention includes a case . 
where program code, read from a memory medium, is 
written in a memory that is mounted on a function ex- 
pansion board inserted into a computer, or In a function 
expansion unit connected to a computer, and in conso- 
nance with a program code instruction, a CPU mounted 
on the function expansion board or the function expan- 
sion unit performs one part, or all of the actual process- 
ing in order to Implement the functions In the above de- 
scribed embodiments. 

When the present invention is applied tor the above 
memory medium, program code corresponding to the 
previously described flowcharts Is stored in the memory 
medium. 



Claims 

1 . A remote control apparatus comprising: 

reception means for receiving information con- 
cerning a device to be controlled; 
validation means for validating operating pro- 
cedures for said device to be controlled; and 
transmission control means for transmitting an 
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operating command to said device based on 
said operating procedures that are validated. 

2. A remote control apparatus according to claim 1 , 
wherein said transmission control means issues a 
request, to said device to be controlled, tor trans- 
mission of information concerning said device. 

3. A remote control apparatus according to claim 1, 
wherein said information received by said reception 
means consists of operating procedures for said de- 
vice that is to be controlled. 

4. A remote control apparatus according to claim 1, 
further comprising: 

storage means for storing operating proce- 
dures in consonance with a device to be con- 
trolled, 

wherein said information received by said re- 
ception means constitutes identification means 
for a device to be controlled and said validation 
means validates said operating procedures 
stored in consonance with said identification in- 
formation for said device. 

5. A remote control apparatus according to claim 1, 
further comprising: 

storage means for storing operating proce- 
dures in consonance with a type of a device that 
is to be controlled, 

wherein said information received by said re- 
ception means is for a type of a device to be 
controlled and said validation means validates 
said operating procedures stored in conso- 
nance with said type of said device. 

6. A remote control apparatus according to claim 1, 
wherein said information concerning said device 
that is to be controlled is received from said device 
that is to be controlled. 

7. A remote control apparatus according to claim 1, 
wherein said Information concerning sard device to 
be controlled is received from a device other than 
said device that is to be controlled. 

8. A remote control apparatus according to claim 1, 
wherein said reception means receives data by 
reading shape information for a specific portion of 
said device to be controlled, and said validation 
means selects a device corresponding to said 
shape information, and validates operating proce- 
dures for said device that is selected. 

9. A remote control apparatus according to claim 8, 
wherein said shape information is for an entire de- 



vice that is to be controlled. 

10. A remote control apparatus according to claim 8. 
wherein said shape informatbn is bar code at- 

5 tached to a device that is to be controlled. 

11. A remote control apparatus according to claim 1, 
wherein said reception means and transmission 
control means are employed to perform bidirection- 

^0 ai communication with said device that is to be con- 
trolled. 

12. A remote control apparatus according to claim 11, 
wherein said reception means and said transmis- 

^5 sion control means are capable of using a plurality 
of communication routes. 

13. A remote control apparatus according to claim 11, 
further comprising selection means for selecting a 

20 specific communication route from among said plu- 
rality of communication routes. 

14. A rerrx>te control apparatus according to claim 11, 
further comprising selection means for selecting a 

25 second communication route while a first commu- 
nication route is currently selected from among said 
plurality of communication routes. 

15. A renriote control apparatus according to claim 11, 
30 wherein said plurality of communication routes are 

all to be employed at the same time. 

16. A remote control apparatus according to claim 1, 
further comprising storage means for storing a his- 

35 tory of operations performed according to said op- 
erating procedures. 

17. A remote control apparatus according to claim 1, 
wherein said operating command is transmitted to 

40 a device other than said device that is to be control- 
led, and is transmitted through said device to said 
device to be controlled. 

18. A remote control apparatus according to claim 1, 
45 wherein said operating command is transmitted to 

a first device that is to be controlled, and is trans- 
mitted to a second device that is to be controlled 
through said first device that is to be controlled. 

50 19. A remote control apparatus according to claim 1, 
wherein the status of said device that is to be con- 
trolled is received by said reception means and said 
operating procedures are changed according to 
said status. 

55 

20. A remote system comprising: 

identification means for identifying a device to 
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be controlled by a remote control apparatus; 
and 

transmission control means for transmitting to 
said remote control apparatus information con- 
cerning a device that is identified, 
wherein said remote control apparatus trans- 
mits an operating command to said device in 
accordance with operating procedures based 
on said information that is received. 

21. A remote control system according to claim 20, 
wherein said information concerning said device 
that is to be controlled consists of operating proce- 
dures (or said device, 

22. A remote ccnirol system according to claim 20, 
wherein sa»c iniofmaiior concerning said device 
that is to bo cortro icc consists of identification in- 
formation, and wherein satd rcnxDlc control appara- 
tus employs shO operriKng procedures in conso- 
nance with srtiO iJovicu btoieO in t>rtKJ i emote control 
apparatus. 

23. A remote control system according to claim 20, 
wherein said information concerning said device 
that is to be controlled concerns a type of device 
that is to be controlled, and wherein said remote 
control apparatus employs said operating proce- 
dures in consonance with said device type stored 
in said remote control apparatus 

24. A remote control system according to claim 20, 
wherein said identification means detects a position 
of said remote control apparatus, and employs said 
position and positions of a plurality of devices that 
are to be controlled by said remote control appara- 
tus to identify a device that is to be controlled. 

25. A remote control system according to claim 20, 
wherein said identification means is provided for 
each of devices that are to be controlled by said re- 
mote control apparatus. 

26. A remote control system according to claim 20, 
wherein said identification means identifies an op- 
erator of said remote control apparatus, and desig- 
nates operating procedures in consonance with 
said identity of said operator. 

27. A remote control system according to claim 20, 
wherein said identification means identifies a skill 
level of an operator of said remote control appara- 
tus, and, in consonance with said skill level of said 
operator, designates functions that are to be includ- 
ed in operating procedures. 

28. A remote control system according to claim 20, 
wherein each individual device that is to be control- 



led by said remote control apparatus includes stor- 
age means for storing a history of operations per- 
formed by said device. 

5 29. A remote control system according to claim 28, 
wherein said devices that is to be controlled is op- 
erated by referring to said history of said operations 
performed by said device that is stored in said stor- 
age means. 

10 

30. A remote control system according to claim 20, fur- 
ther comprising storage means for storing a history 
of operations performed by each individual device 
that is to be controlled by said remote control appa- 

15 ratus. 

31. A remote control system according to claim 30, 
wherein said devices that are to be controlled are 
operated by referring to said histories ot said oper- 

20 ations performed by said devices stored in said stor- 
age means. 

32. A remote control apparatus comprising: 

25 input means for receiving an instruction; 

processing means for performing processing in 
accordance with said instruction input by said 
input means; 

communication means for communicating with 

30 another device; and 

communication control means for permitting 
said communication means to transmit an op- 
eration instruction, which has been input for an- 
other device by said input means, to said device 

35 to be controlled according to said instruction. 

33. A remote control apparatus according to claim 32, 
further comprising: 

40 validation means for validating operating pro- 

cedures for a device that is to be controlled ac- 
cording to said operating instruction; and 
input control means tor controlling said input 
means so as to input operation instructions for 

45 said device in accordance with said operating 

procedures that are validated. 

34. A remote control apparatus according to claim 33, 
further comprising: 

50 

storage means for storing operating proce- 
dures for another device, 
wherein said validation means validates said 
operating procedures stored in said storage 
55 means. 

35. A remote control apparatus according to claim 33, 
wherein said reception means receives operating 
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procedures for another device, and said validation 
means validates said operating procedures. 

36. A remote control apparatus according to clainn 35, 
wherein said communication means receives, from 
a device thai is to be controlled, operating proce- 
dures for said device. 

37. A remote control apparatus according to claim 32, 
wherein said communication means receives re- 
sults obtained by processing said device according 
to said operation instruction, and said processing 
means processes said results. 

38. A remote control apparatus according to claim 32, 
wherein results obtained by said processing means 
are transmitted, to said communication means, as 
an object to be processed by said device that is to 
be controlled. 

39. An information processing apparatus comprising: 

determination means for specifying a process 
to be performed; and 

processing means for performing said process 
specified by said determination means. 

40. An information processing apparatus according to 
claim 39, wherein, for a determination, said deter- 
mination means employs at least one part of a proc- 
ess that is to be executed by a locally own appara- 
tus, a status of said apparatus, an instruction from 
a user, and the contents of information that is to be 
processed. 

41. An information processing apparatus according to 
claim 39, wherein said process includes communi- 
cation with an external device. 

42. An information processing apparatus according to 
claim 41 , wherein said process includes an acqui- 
sition of information from said external device. 

43. An information processing apparatus according to 
claim 42, wherein said acquisition of information In- 
cludes a transmission request submitted to said ex- 
ternal device. 

44. An information processing apparatus according to 
claim 41, wherein said communication includes 
storage of information in said extemai device. 

45. An information processing apparatus according to 
claim 41 , wherein said communication includes an 
instruction to said external device. 

46. An information processing apparatus according to 
claim 41 , wherein said communication includes a 



response to a message received from said external 
device. 

47. An information processing apparatus according to 
5 claim 46, wherein said message is a transmission 

request, and information requested is transmitted 
as said response. 

48. An information processing apparatus according to 
10 claim 46, wherein said message is a request for 

storage, and whether or not a storage is available 
is transmitted as said response. 

49. An information processing apparatus according to 
15 claim 46, wherein said message is an instruction, 

and results obtained according to said instruction 
are transmitted as said response. 

50. An information processing apparatus comprising: 

20 

communication means for communicating with 
an externa! device; 

processing means for processing information 
from said external device that is received 
25 through said communication means; 

acquisition means for acquiring information 
from said external device through said commu- 
nication means; and 

control means for permitting said processing 
30 means to process said information acquired by 

said acquisition means. 

51. An information processing apparatus according to 
claim 50, wherein said processing means is output . 

35 means for outputting Information. 

52. An information processing apparatus according to 
claim 50, wherein said acquisition means includes 
request means for issuing a transmission request 

40 to said extemai device, and control means for ena- 
bling reception, through said communication 
means, of information that is transmitted by said ex- 
ternal device upon receipt of said transmission re- 
quest. 

45 

53. An information processing apparatus according to 
claim 50, wherein said acquisition means acquires 
specific information. 

so 54. An information processing apparatus according to 
claim 50, wherein said acquisition means acquires 
information from a specific external device. 

55. An information processing apparatus according to 
55 claim 50, further comprising: 

determination means for determining whether 
information should be acquired by said acqui- 
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sition means, 

wherein said acquisition means acquires infor- 
mation in accordance with results obtained by 
said determination means. 

56. An information processing apparatus according to 
claim 55, further comprising: 

request input means for inputting an output re- 
quest from a user, 

wherein said determination means decides that 
information is to be acquired upon receipt of 
said output request. 

57. An information processing apparatus according to 
claim 56, wherein said request input means in- 
cludes: 

natural language input means for inputting in- 
formation using a natural language; and 
analysis means for analyzing said information 
contained in said natural language input by said 
natural language input means and for identify- 
ing said output request. 

58. An information processing apparatus according to 
claim 55, further comprising: 

identification means for identifying a status of 
said apparatus, 

wherein said determination means determines 
that information is to be acquired in accordance 
with said status. 

59. An information processing apparatus according to 
claim 51 , wherein said output means prints informa- 
tion. 

60. An information processing apparatus according to 
claim 51 , wherein said output means displays infor- 
mation. 

61. An information processing apparatus according to 
claim 51 , wherein said output means outputs infor- 
mation by voice. 

62. An information processing apparatus according to 
claim 52, wherein said transmission request in- 
cludes information for designating an information 
format. 

63. An information processing apparatus according to 
claim 50, further comprising: 

conversion means for changing a format for 
information received via said communication 
means. 

64. An information processing apparatus according to 



claim 52, wherein said transmission request is a re- 
quest for a list of information. 

65. An information processing apparatus according to 
5 claim 52, wherein information to be transmitted is 

electronic mail. 

66. An information processing apparatus comprising: 

10 communication means for communicating with 

an external device; 

processing means for processing information 
from said external device that is received 
through said communication means; 

75 transmission control means for transmitting to 

said external device through said communica- 
tion means said information that is processed 
by said processing means; and 
control means for storing in said external device 

20 said information that has been transmitted to 

said external device through said communica- 
tion means. 

67. An information processing apparatus according to 
25 claim 66, wherein said processing means is input 

means for inputting information. 

68. An information processing apparatus according to 
claim 66, wherein said transmission control means 

30 comprises a request means for issuing to said ex- 
ternal device a reception request for information, 
and wherein, when said reception request is ac- 
cepted, said transmission control means transmits 
information that has been input by said input means 

35 to said external device through said communication 
means. 

69. An information processing apparatus according to 
claim 66, wherein said transmission control means 

40 transmits information to a specific external device. 

70. An Information processing apparatus according to 
claim 66, wherein said transmission control means 
transmits information to a plurality of external devic- 
es es. 

71. An information processing apparatus according to 
claim 68, wherein said transmission control means 
transmits said reception request in parallel to a ptu- 

50 rality of external devices. 

72. An information processing apparatus according to 
claim 66, further comprising: 



55 determination means for determining whether 

a transmission to an external device should be 
performed, 

wherein said transmission control means con- 
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trols said transmission in accordance with re- 
sults obtained by said determination means. 

73. An information processing apparatus according to 
claim 72. further comprising: 

request input means for inputting a transmis- 
sion request from a user, 
wherein said determination means determines 
that a transmission should be performed in ac- 
cordance with said transmission request. 

74. An information processing apparatus according to 
claim 72 further comprising: 

idenliMCrilion means tor identifying a status, 
wherein said dcicrmination means makes a de- 
termination in accordance with said status. 



83. An informatbn processing apparatus according to 
claim 76, wherein a process that is to be performed 
based on said instruction information is an informa- 
tion printing process. 

5 

84. An informatbn processing apparatus according to 
claim 76, wherein a process performed based on 
said instruction information is an information display 
process. 

TO 

85. An information processing apparatus according to 
claim 76, wherein a process performed based on 
said instruction information is an information stor- 
age process. 

75 

86. An information processing apparatus according to 
claim 67, wherein said input means is means tor 
reading image information. 



75. An informddon processing apparatus according to 
claim 66, wheioirt ba»d transmission control means 
transmits said intofmation thai has been input, in 
parallel to a plurality of external devices. 



82. An information processing apparatus according to 
claim 76, wherein said information includes infor- 
mation for which processing is to be performed by 
said external device in accordance with said in- 
struction information. 



20 87, An information processing apparatus according to 
claim 67, wherein said input means is means for 
forming an image from image information. 

.88. An information processing apparatus comprising: 

operating means for performing an operation; 
input means for inputting information; 
processing means for, based on said operation, 
processing said information input by said input 
means; 

output means for outputting results obtained by 
said processing means; and 
control means for controlling another device 
based on an operation for said device per- 
formed by said operation means. 

89. An informatbn processing apparatus according to 
ciaim 88, further comprising: 

communication means for communicating with 
another device, 

wherein said control means permits said com- 
munication means to transmit instruction infor- 
mation corresponding to an operation per- 
formed by said operating means. 

90. An informatbn processing apparatus according to 
claim 88, wherein said operating means is capable 
of operating said other device even in a state where 
communication with said device is disabled, and 
wherein said control means transmits said instruc- 
tion information corresponding to said operation 
when communication is enabled. 

SB 91. An informatbn processing apparatus according to 
claim 88, further comprising: 

designation means tor selecting a specific de- 



76. An information processing apparatus according to 25 
claim 66, wherein said information includes instruc- 
tion information concerning said external device. 

77. An information processing apparatus according to 
claim 76. wherein said instruction information in- 30 
eludes setup instruction information for a process 
performed by said external device. 

78. An information processing apparatus according to 
claim 77, wherein said process performed by said 3S 
external device is an information filing process. 

79. An information processing apparatus according to 
claim 77, wherein said process performed by said 
external device is an information search process. 40 

80. An information processing apparatus according to 
claim 76, wherein said instruction information in- 
cludes information for instructing a first external de- 
vice to withdraw information from a second external 45 
device. 

81. An information processing apparatus according to 
claim 76, wherein said instruction information in- 
cludes information for Instructing a first external de- so 
vice to transfer information to a second external de- 
vice. 
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vice from among said plurality ot other devices, 
wherein said control means controls a device 
selected by said designation means. 

92. An information processing apparatus according to 
claim 89, wherein said control means includes com- 
munication control means tor permitting said com- 
munication means to transmit, to said device select- 
ed by said designation means, operating informa- 
tion that specifies an operation that is to be per- 
formed by said operating means. 

93. An information processing apparatus according to 
claim SB. further comprising: 

validation means for validating operating pro- 
cedures tor said other device; and 
oporrtiing control means for controlling said op- 
erating means, so that processing correspond- 
ing to said operating information is to be per- 
loiined in accordance with said validated oper- 
ating procedures. 

94. An intcrmalion processing apparatus according to 
cU'jim 03. further comprising: 

storage means for storing operating proce- 
dures for another device, 

wherein said validation means validates oper- 
ating procedures stored in said storage means. 

95. An information processing apparatus according to 
claim 93. wherein operating procedures for another 
device are received by said communication means, 
and are validated by said validation means. 

96. An information processing apparatus according to 
claim 95, wherein said communication means re- 
ceives, from said device that is to be controlled, said 
operating procedures for said device. 

97. An information processing apparatus according to 
claim 95. wherein said communication means re- 
ceives, from a device other than said device that is 
to be controlled, said operating procedures tor said 
device that is to be controlled. 

98. An information processing apparatus according to 
claim 68, further comprising: 

operating control means for controlling said 
operating means so that operating procedures for 
sard apparatus are employed to perform an opera- 
tion corresponding to operating infornr»ation that 
concerns said device that is to be controlled. 

99. An information processing apparatus according to 
claim 98, wherein said operating information is in- 
struction information tor instructing a process cor- 



responding to said operation performed by said de- 
vice to be controlled. 

100. An information processing apparatus according to 
5 claim 90, wherein said designation means is capa- 
ble ot selecting a plurality of devices that are to be 
controlled at the same time 

101. An information processing apparatus according to 
10 claim 100, wherein said designation means is ca- 
pable of selecting said apparatus as one of a plu- 
rality of devices that are to be controlled at the same 
time. 

15 102.An information processing apparatus according to 
claim 88. further comprising: 

identification means for identifying a plurality ot 
devices that are to be controlled by said control 
20 means; 

detection means for detecting functions of a 
plurality of devices that are to be controlled and 
that are identified by said identification means; 
and 

25 activation means for activating said functions 

that are detected by said detection means. 

103. An information processing apparatus according to 
claim 102, further comprising: 

30 notification means for issuing an activation 

notification for said functions activated by said ac- 
tivation means- 

104. An information processing apparatus according to 
35 claim 103, further comprising: 

requesting means for requesting that a user 
issue a confirmation before activation of said func- 
tions is performed by said activation means. 

40 105. An information processing method comprising: 

a determination step of specifying a process to 
be performed by an information processing ap- 
paratus; and 

45 a processing step of performing said process 

specified at said determination step. 

106. An information processing method according to 
claim 105, wherein, for a determination, at said de- 

50 termination step employed is at least one part of a 
process that is to be executed by a locally own ap- 
paratus, a status of said apparatus, an instruction 
from a user, and the contents of information that is 
to be processed. 

55 

107. An infomnation processing method according to 
claim 105, wherein said process includes commu- 
nication between an external device and said infor- 
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mation processing apparatus. 

108. An information processing nnethod according to 
claim 107, wherein said process includes an acqui- 
sition of information from said extemal device. 5 

109. An information processing method according to 
claim 108, wherein said acquisition of information 
includes a transmission request submitted to said 
external device. io 

110. An information processing method according to 
claim 107, wherein said communication includes 
storage of information in said external device. 

15 

111. An information processing method according to 
claim 107, wherein said communication includes an 
instruction to said external device. 

11 2. An information processing method according to 
claim 107, wherein said communication includes a 
response to a message received from said external 
device. 

IIS.Ar^ information processing method according to 
claim 112, wherein said message is a transmission 
request, and information requested is transmitted 
as said response. 

11 4. An information processing method according to 30 
claim 112, wherein said message is a request for 
storage, and whether or not a storage is available 

is transmitted as said response. 

115. An information processing method according to 3S 
claim 112, wherein said message Is an instruction, 
and results obtained according to said instruction 
are transmitted as said response. 

IIS.An information processing method comprising: 40 

a processing step of processing information 
that is received from an external device through 
a communication unit, which is used to commu- 
nicate with said external device; 4S 
an acquisition step of acquiring information 
through said communication unit from said ex- 
ternal device; and 

a control step of, at said processing step, 
processing said information that is acquired at 50 
said acquisition step. 

11 7. An information processing method according to 
claim 116, wherein said processing step is an output 
step of outputting information. ss 

11 8. An information processing method according to 
claim 116, wherein said acquisition step includes a 



request step of issuing a transmission request to 
said external device, and a control step of enabling 
reception, through said communication unit, of in- 
formation that is transmitted by said external device 
upon receipt of said transmission request. 

119. An information processing nnethod according to 
claim 116. wherein specific information is acquired 
at said acquisition step. 

120. An information processing method according to 
claim 116, wherein information is acquired from a 
specific external device at said acquisition step. 

121. An information processing method according to 
claim 116, further comprising: 

a determination step of determining whether in- 
formation should be acquired at said acquisi- 
tion step, 

wherein, at said acquisition step, information is 
acquired in accordance with results obtained at 
said detemnination step. 

122. An information processing method according to 
claim 121, further comprising: 

a request input step of inputting an output re- 
quest from a user, 

wherein at said determination step it is decided 
that information is to be acquired upon receipt 
of said output request. 

123. An information processing method according to 
claim 1 21 , wherein said request input step includes: 

a natural language input step of inputting infor- 
mation using a natural language; and 
an analysis step of analyzing said Infonmation 
contained in said natural language input at said 
natural language input step and of identifying 
said output request. 

124. An information processing method according to 
claim 121, further comprising: 

an identification step of identifying a status of 
said apparatus, 

wherein at said determination step it is ascer- 
tained that information is to be acquired in ac- 
cordance with said status. 

125. An information processing method according to 
claim 115, wherein said output step is an informa- 
tion printing process. 

126. An information processing method according to 
claim 115, wherein said output step is an informa- 
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tion display process. 

127. An information processing method according to 
claim 115, wherein said output step is an informa- 
tion output process by voice. 

128. An information processing method according to 
claim 118, wherein said transmission request in- 
cludes informatbn for designating an information 
format. 

129. An information processing method according to 
claim 118, further comprising: 

a conversion step of changing a format for in- 
formation received via said communication unit. 

130. An information processing method according to 
claim 118, wherein said transmission request is a 
request for a list of information. 

131. An information processing method according to 
claim 118, wherein information to be transmitted is 
electronic mail. 

132. An information processing method comprising: 

a processing step of processing information; 
a transmission step of transmitting said infor- 
mation that has been processed at said 
processing step to an e)cternal device through 
a communication unit, which is used to commu- 
nicate with said external device; and 
a control step of storing in said external device 
said information that has been transmitted 
through said communication unit to said exter- 
nal device. 

133. An information processing method according to 
claim 132, wherein at said processing step is an in- 
put step of inputting information. 

134. An information processing method according to 
claim 132, wherein said transmission control step 
comprises a request step of issuing to said external 
device a reception request for information, and 
wherein, when said reception request is accepted, 
at said transmission control step, information that 
has been input at said input step is transmitted to 
said external device through said communication 
unit. 

135. An information processing method according to 
claim 132, wherein at said transmission control step 
information is transmitted to a specific external de- 
vice. 

136. An information processing method according to 
claim 1 32, wherein at said transmission control step 



information is transmitted to a plurality of external 
devices. 

137. An information processing method according to 
5 claim 134, wherein at said transmission control 

step, said reception request is transmitted in paral- 
lel to a plurality of external devices. 

138. An information processing method according to 
10 claim 132, further comprising: 

a determination step of determining whether a 
transmission to an external device should be 
performed, 

15 wherein, at said transmission control step, said 

transmission is controlled in accordance with 
results obtained at said determination step. 

139. An information processing method according to 
20 claim 138, further comprising: 

a request input step of inputting a transmission 
request from a user, 

wherein, at said determination step, it is ascer- 
25 tained that a transmission should be performed 

in accordance with said transmission request. 

140. An infonnation processing method according to 
claim 138, further comprising: 

30 

an identification step of identifying a status, 
wherein, at said detemnination step, a determi- 
nation is made in accordance with said status. 

35 141 .An information processing method according to 
claim 132, wherein at said transmission control 
step, said Information that has been input is trans- 
mitted in parallel to a plurality of external devices. 

40 142. An information processing method according to 
claim 132, wherein said information includes in- 
struction information concerning said external de- 
vice. 

45 i43.An information processing method according to 
claim 142, wherein said instruction information in- 
cludes setup instruction information for a process 
performed by said external device. 

50 l44.An information processing method according to 
claim 143, wherein said process performed by said 
external device is an information filing process. 

145. An information processing method according to 
55 claim 143, wherein said process performed by said 

external device is an information search process. 

146. An infonnation processing method according to 
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claim 142, wherein said instruction information in- 
cludes information lor instructing a first external de- 
vice to withdraw information from a second external 
device. 

147. An information processing method according to 
claim 142, wherein said instruction information in- 
cludes information for instructing a first external de- 
vice to transfer information to a second external de- 
vice. 

148. An information processing method according to 
claim 142 wherein said information includes infor- 
mation lor which processing is to be performed by 
said external device in accordance with said in- 
struction intormation. 

149. An inlormation processing method according to 
claim 142 wTierein a process that is to be performed 
based on said instruction information is an informa- 
tiofi printing process. 

150. An in(orrT»alion processing method according to 
claim 142 wherein a process performed based on 
said instruction inlormation is an information display 
process 

151 .An information processing method according to 
claim 142 wherein a process performed based on 
said instruction information is an information stor- 
age process 

152. An information processing method according to 
claim 142, wherein said input step is a step of read- 
ing image information. 

153. An information processing method according to 
claim 142, wherein said input step is a step of form- 
ing an image from image information. 

154. An information processing method, for an informa- 
tion processing apparatus that has an operating unit 
for performing processing, comprising: 

an input step ol inpuUing information; 

a processing step of processing, based on said 

operation, said information input at said Input 

step; 

an output step of outputting results obtained at 
said processing step; and 
a control step of controlling another device 
based on an operation performed for said de- 
vice by said operation unit. 

155. An information processing method according to 
claim 154. further comprising: 

a communication step of communicating with 



another device, 

wherein, at said control step, said communica- 
tion means is permitted to transmit instruction 
information corresponding to an operation per- 
5 formed at said operating step. 

156. An information processing method according to 
claim 154, wherein said operating unit is capable of 
operating said other device even in a state where 
^0 communication with said device is disabled, and 
wherein at said control step said instruction infor- 
mation corresponding to said operation is transmit- 
ted when communication is enabled, 

^5 l57.An information processing method according to 
claim 154, further comprising: 

a designation step of selecting a specific device 
from among said plurality of other devices, 
wherein, at said control step, a device selected 
at said designation step is controlled. 

158. An information processing method according to 
claim 154, wherein said control step includes a 

25 communication control step of permitting said com- 

munication unit to transmit, to said device selected 
at said designation step, operating information that 
specifies an operation that is to be performed by 
said operating unit. 

30 

159. An Information processing method according to 
claim 154, further comprising: 

a validation step of validating operating proce- 
dures for said other device; and 
an operating control step of controlling said op- 
erating means, so that processing correspond- 
ing to said operating information is to be per- 
formed in accordance with said validated oper- 
"^0 atlng procedures. 

160. An information processing method according to 
claim 1 59, wherein at said validation step, validated 
are operating procedures, which are stored in a 

45 storage unit for storing operating procedures for an- 
other device. 

161 .An information processing method according to 
claim 159, further comprising: 

50 

an operation procedure receptipn unit of receiv- 
ing operating procedures for another device 
through said communication unit, 
wherein said received operating procedures 
are validated by said validation means. 

162. An information processing method according to 
claim 161, wherein at said communication step. 
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said operating procedures for said device are re- 
ceived from said device that is to be controlled. 

163. An information processing method according to 
claim 161, wherein at said communication step, 
said operating procedures for said device that is to 
be controlled are received from a device other than 
said device that is to be controlled. 

164. An information processing method according to 
claim 154, further comprising: 

an operating control step of controlling said 
operating unit so that operating procedures tor said 
apparatus are employed to perform an operation 
corresponding to operating information that con- 
cerns said device that is to be controlled. 

165. An information processing method according to 
claim 164, wherein said operating information is in- 
struction information tor instructing a process cor- 
responding to said operation performed by said de- 
vice to be controlled. 

166. An information processing method according to 
claim 156, wherein at said designation step, it is 
possible to select a plurality of devices that are to 
be controlled at the same time. 

167. An information processing method according to 
claim 166, wherein at said designation step, it is 
possible to select said apparatus as one of a plural- 
ity of devices that are to be controlled at the same 

.time. 

168. An information processing method according to 
claim 154, further comprising: 

an identification step of identifying a plurality ot 
devices that are to be controlled at said control 
step; 

a detection step of detecting functions of a plu- 
rality of devices that are to be controlled and 
that are identified at said identification step; and 
an activation step of activating said functions 
that are detected at said detection step. 

169. An information processing method according to 
claim 168, further comprising: 

a notification step of issuing an activation no- 
tification for said functions activated at said activa- 
tion step. 

170. An information processing method according to 
claim 169, further comprising: 

a requesting step of requesting that a user is- 
sue a confirmation before activation of said func- 
tions is performed at said activation step- 
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171 .A computer-readable storage medium used to store 
an information processing program for controlling a 
computer for processing of information, said pro- 
gram comprising codes for enabling said computer 
to perform: 

a determination step of specifying a process to 
be perfomned by an information processing ap- 
paratus; and 

a processing step of performing said process 
specified at said determination step. 



172. A computer-readable storage medium used to store 
an information processing program for controlling a 
IS computer for processing of information, said pro- 
gram comprising codes for enabling said computer 
to perform: 



a processing step of processing information 
20 that is received from an external device through 

a communication unit, which is used to commu- 
nicate with said external device; 
an acquisition step of acquiring information 
through said communication unit from said ex- 
ternal device; and 

a control step of, at said processing step, 
processing said information that is acquired at 
said acquisition step. 

173. A computer-readable storage medium used to store 
an information processing program tor controlling a 
computer for processing of information, said pro- 
gram comprising codes for enabling said computer 
to perform: 

a processing step of processing information; 
a transmission step of transmitting said infor- 
mation that has been processed at said 
processing step to an external device through 
a communication unit, which is usedto commu- 
nicate with said external device; and 
a control step of storing in said external device 
said information that has been transmitted 
through said communication unit to said exter- 
nal device. 

174. A computer-readable storage medium usedto store 
an information processing program for controlling a 
computer for processing of information, said pro- 

50 gram comprising codes for enabling said computer 
to perform: 

an input step of inputting information; 
a processing step of processing, based on said 
55 operation, said information input at said input 

step; 

an output step of outputting results obtained at 
said processing step; and 
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a control step of controlling another device 
based on an operation performed for said de- 
vice by said operation unit. 
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CANON 

DATE: NOV 25, 1996 

TO: MACROHARD CORP. 
ATTN: MR. JOHN SMITH 
TEL: 001-1-{415)-394-11l1 
FAX: 001-1-(415)-394-2312 

FROM: CANON INC. 
SENDER: DR. ROHRA 
TEL: +81-44-543-3333 
FAX: +81-44-543-5555 



DEAR SIR, 

THANK YOU FOR YOUR PHONE CALL THIS MORNING. 

PLEASE FIND ENCLOSED HERE WITH THE INFORMATION REGARDING THE 
HOTEL ACCOMMODATION AND OTHER TRAVEL INFORMATION FOR YOUR 
FORTH COMING TRIP TO JAPAN. I WILL BE COMING TO YOKOHAI^ C'Jl.J'" 
TERMINAL TO PICK YOU UP, HENCE KINDLY CALL ME AT MY RESIDENCE. 
AFTER YOU BOARD THE LIMOUSINE BUS. 

HOTEL ACCOMMODATION FOR YOUR TRIP IS AS FOLLOWS: 
HOTEL: SHIN-YOKOHAMA PRINCE HOTEL 
PERIOD OF STAY: 18TH DEC 1996 - 22ND DEC 1996 (4 NIGHTS) 



HAVE A PLEASANT JOURNEY. 
WITH REGARDS, 



(ARUNA ROHRA) 
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FIG. 92 



DATE: NOV 22, 1996 




TO: CANON INC. 
ATTN: DR. ARUNA ROHRA 
TEL: +81-44-543-3333 
FAX: +81-44-543-5555 


FROM: MACROHARD CORP. 
SENDER: MR. JOHN SMITH 
TEL: (415)-394-1111 
FAX: (415)-394-2312 


DEAR DR. ROHRA, 




THANK YOU FOR YOUR FAX. 




1 WILL BE REACHING YOKOHAMA AT 3:00 PM ON 28TH NOVEMBER. 
SINCE 1 AM COMING FOR THE FIRST TIME TO YOKOHAMA AND AS 1 AM NOT VERY 
FLUENT IN JAPANESE, 1 WOULD BE GRATEFUL IF YOU COULD KINDLY MEET ME AT 
THE STATION. 


THANKING YOU. 




YOURS SINCERELY. 




JOHN SMITH 
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ORGANIZATION: CANON INC. 

TO: tanaka@dos.canon.co.jp 

REFERENCES: 



MR JOHN ARRIVES AT YOKOHAMA ON 28TH. ROHRA WOULD LIKE 
YOU TO MEET HIM AND TAKE CARE OF HIM. 



I ATTACH THE FACSIMILE FROM HIM FOR YOUR REFERENCE. 
THANK YOU. 



-YOUR FAVOURITE FRIEND! ;-) 



BNSDOCID: <EP ^0853301 A2J_> 



143 



EP 0 853 301 A2 



FIG. 95 



NO 



YES 



( START ) 



RECOGNIZE INPUTTED 
DOCUMENT 



S950 



USER HAS TO DO ANY 
ACTION ? 



r YES 



PROXY MAY EXECUTE 
ACTION ? 



u NO 



ANY USER ? 



NO 



EXECUTION OF ACTION 
REQUIRED ? 



NO 



C END ) 



S951 



S952 



S953 
YES 



1 



S954 



INQUIRE USER 



S955 
YES 



1 



S956 



EXECUTE ACTION 



BNSDOCID: <EP 0853301A2_I_> 



144 



EP 0 853 301 A2 




145 



BNSOOCID; <EP ^0853301 A2J_> 



EP 0 853 301 A2 




BNSCXX^ID: <EP ^085330 1A2_L> 



146 



EP 0 853 301 A2 



FIG. 98 





fpT \ 


CD 


r . L 






SELECT 

DOCUMENT TO 
BE PRINTED 






DETERMINE 
NUMBER OF 
COPIES 

[11 COPY V 


1 

SELECT 
DESTINATION 

A CO. MR. a 
B CO. MR. b 
C CO. MR. c 









147 



BNSDOCID: <EP ^0853301 A2_l_> 



EP 0 853 301 A2 



FIG. 99 

C START ~^ 




SIGNAL OF TARGET UNIT 
IDENTIFICATION SIGNAL 




8994 



DISPLAY UI CORRESPONDING 
TO IDENTIFICATION SIGNAL 



t 

C END 



148 



EP 0 853 301 A2 



FIG. 100 




TRANSMIT TARGET UNIT 
IDENTIFICATION SIGNAL 



( END ) 



149 



EP 0 853 301 A2 



FIG. 101 



961 







1 


CD 





^/^ELECT ^ 
DOCUMENT TO 
BE PRINTED 
FILE A 
FILE B 
^ FILE 0 > 




(1)ETERMINE^ 
NUMBER OF 
COPIES 

{11 COPY V 



^SELECT 
DESTINATION 

A CO. MR. a 
B CO. MR. b 
V ^CO. MR. c 



961 



/Select ^ 
dxument to 
be printed 

FILE A 
FILE B 
FILE 0 > 




SELECT 
DEVICE 
WHICH YOU 
WISH TO USE 



SELECT 
DOCUMENT 
TO BE 
PRINTED 

FILE A 
FILE B 
FILE C 





DETERMINE 
NUMBER OF 
COPIES 

[11 COPY # 




/^SELECT 
DESTINATION 

A CO. MR. a 
B CO. MR. b 
C CO. MR. c 



DETERMINE 
NUMBER OF, 
COPIES 



[11 COPY $ 




ISO 



BNSDOCID: <EP ^0853301A2_I_> 



EP 0 853 301 A2 



FIG. 102 



C START ) 




TRANSMIT UI REQUEST 
SIGNAL 




DISPLAY RECEIVED UI 



( END ) 



EP 0 853 301 A2 



FIG. 103 



C START ^ 




TRANSMIT UI 



( END ) 



BNSDOCID: <EP ^0853301 A2_L^ 



152 



EP 0 853 301 A2 



FIG. 104 




OS 



TO: REMOCON A 

DETERMINE NUMBER 
OF COPIES 

[1]C0PY $ 



HERE 
REMOCON A. 
SEND UI 



SELECT 
DEVICE 
WHICH YOU 
WISH TO USE 




DETERMINE 
NUMBER OF COPIES 

(1) COPY # . 



DETERMINE 
NUMBER OF. 
COPIES 



[1] COPY 0 




-963 



153 



EP 0 853 301 A2 



FIG. 105 



( START ) 




DISPLAY RECEIVED UI 



( END ) 



154 



EP 0 853 301 A2 



FIG. 106 



( START ) 




TRANSMIT REMOCON 
IDENTIFICATION INFORMATION 
AND TARGET UNIT IDENTIFICATION 
INFORMATION TO FILING SERVER 



( END ) 



155 



EP 0 853 301 A2 



FIG. 107 



C START ) 




TRANSMIT UI AS SHOWN IN TARGET 
UNIT IDENTIFICATION SIGNAL TO 
REMOCON AS SHOW IN REMOCON 
IDENTIFICATION INFORMATION 



( END ) 



156 



EP 0 853 301 A2 



FIG. 108 





TO: REMOCON B ^ 
SELECT DESTINATION 
A CO. MR. a 
B CO. MR. b 



MULTI-ADDRESS 
COMMUNICATION 



SELECT 
DEVICE 
WHICH YOU< 
WISH TO USE 




riO: REMXON A 
SELECT 
DESTINATION 
A CO. MR. a 
B CO. MR. b 
C CO. MR. c 



HERE BEGINNER 
REMOCON A. 
SEND UI. 



HERE SENIOR 
REMOCON B. 
VSENDUI. ^ 



960a 



SELECT 
DEVICE 
WHICH YOU 
WISH TO USE 




960b 



/^SELECT DESTINATION ^ 
A CO. MR. a 
B CO. MR. b 



MULTI-ADDRESS 
COMMUNICATION 




SELECT DESTINATION) 
A CO. MR. a 
B CO. MR. b 
C CO. MR. c 



SELECT 
DESTINATI 



A CO. MR. a 
B CO. MR. b 



MULTI-ADDRESS 
COMMUNICATION 




157 



EP 0 853 301 A2 



FIG. 109 



C START ^ 




TRANSMIT REMOCON 
IDENTIFICATION INFORMATION 
USER IDENTIFICATION INFORMATION 
AND UI REQUEST SIGNAL 



S1092 




C END ) 



158 



EP 0 853 301 A2 



FIG. 110 



C START ) 




TRANSMIT REMOCON IDENTIFICATION 
INFORMATION. USER IDENTIFICATION 
INFORMATION. AND TARGET UNIT 
IDENTIFICATION INFORMATION TO 
FILING SERVER 



][ 

C END ) 



BNSDCXID: <EP ^0853301 A2J_> 



159 



EP 0 853 301 A2 



FIG. 111 



C START ^ 




TRANSMIT UI AS SHOWN IN TARGET U 
UNIT IDENTIFICATION INFORMATION 
AND USER IDENTIFICATION INFORMATION 
TO REMOCON AS SHOWN IN REMOCON 
IDENTIFICATION INFORMATION 



S1112 



( END ^ 



F/G. 112 





BEGINNER 


SENIOR 


FAX 


STANDARD UI 


STANDARD UI 
+MULTI-ADDRESS 
COMMUNICATION UI 



160 



EP 0 853 301 A2 



FIG. 113 




963 



TO: REMXON A 
SELECT DESTINATION 

A CO. MR. a 
B CO. MR. b 
C CO. MR. c 



HERE REMOCON A 
MAKE READY FOR FAX 
.COMMUNICATION I 
> 



SELECT 
FUNCTION 
THAT YOU 
WISH TO DO. 

COPY/ 



PRINT 




.960 



SELECT DESTINATION 
A CO. MR. a 
B CO. MR. b 

V C CO. MR. c 




161 



EP 0 853 301 A2 



FIG. 114 




1041 



TRANSMIT 
DOCUMENT 
TO MR. a ! 




MAKE READY 
FOR TRANSMISSION OF 
^DOCUMENT TO MR. a ! 

^ — - 



960 



SELECT 

DESTINATION , 

I A GO. MR. a<Y 
B CO. MR. b 
C CO. MR. c 




BNSOCX^ID: <EP 0853301A2_L> 



162 



EP 0 853 301 A2 



FIG. 115 




11) COPY 0 



163 



EP 0 853 301 A2 



FIG. 116 

( START ) 




SEND REMOCON ID INFORMATION 
AND UI REQUEST SIGNAL 



S1162 




DISPLAY RECEIVED UI 



S1 166 



81164 
NO 



SEND REMOCON ID 
INFORMATION AND UI 
REQUEST SIGNAL VIA 
ANOTHER ROUTE 



-81165 



( END ) 



164 



EP 0 853 301 A2 



FIG. 117 




TOM'S DOCUMENT 
HAS BEEN TRANMITTED 
TO MR. a 



K^962 Un^° 




MR. TOM TRANSMIHED 
FILE ABC TO MR. a OF 
A CO., LTD AT 10:00 AM 
I jDN JULY 5. 1996. ^ 

FAX DXUMENT 



I AM TOM. 
TRANSMIT 
DOCUMENT 
TO MR. a I 



TO: A CO. MR. a 

WE HAVE BEEN ORDERED 
REGARDING THE MAHER. 

TOM 




.963 



BNSDOCID: <EP ^0853301A2_L> 



165 



EP 0 853 301 A2 



FIG. 118 



C START ) 




INFORMATION. USER IDENTIFICATION 
INFORMATION AND REQUEST SIGNAL 
CORRESPONDING TO INSTRUCTIONS 




DISPLAY RECEIVED UI 



C END ) 



BNSDOCID: <EP 0853301A2J_> 



166 



EP 0 853 301 A2 



FIG. 119 



( START ) 




EXECUTE PROCESS 
CORRESPONDING TO 
REQUEST SIGNAL 



-S1192 



TRANSMIT REMOCON IDENTIFICATION 
INFORMATION, USER IDENTIFICATION 
INFORMATION, TARGET UNIT 
IDENTIFICATION INFORMATION, AND 
IMPORTANT INFORMATION RELATED 
TO EXECUTED ACTION TO FILING 
SERVER 



S1193 



( END ) 



167 



EP 0 853 301 A2 



FIG, 120 



( START ^ 




RESERVE RECEIVED 
INFORMATION 



r 



S1202 



UPDATE HISTORY 
INFORMATION 



r 



S1203 



TRANSMIT UI AS SHOWN IN 
TARGET UNIT IDENTIFICATION 
INFORMATION AND USER 
IDENTIFICATION INFORMATION 
TO REMOCON AS SHOWN IN 
REMOCON INFORMATION 



5- 



S1204 



C END ^ 



168 



EP 0 853 301 A2 



FIG. 121 



TIME 


DIRECTOR 


ACTION 


1996/7/5 10:00 


TOM 


TRANSMIT FILE ABC TO A CO. MR. A 


1996/7/3 18:50 


MARY 


TAKE TEN COPIES OF FILE XYZ 



BNSOCCID: <EP ^0853301 A2_L> 



169 



EP 0 853 301 A2 



FIG. 122 






'P^ ' 


1 

i 


— 1 

r 1 



V^961 





MR. TOM TRANSMITTED^ 
FILE ABC TO MR. a OF 
A CO., LTD AT 10:00 AM 
ON JULY 5, 1996. 



FAX DOCUMENT ^ 
TO: A CO. MR. a 

WE HAVE BEEN ORDERED 
REGARDING THE MAHER. 

TOM 




I AM TOM. 
SHOW 
\ ^STORY I 



960 



1, 



HISTORY j yi 
17/5 FILE ABCy f 




170 



EP 0 853 301 A2 



FIG. 123 




3^ 





963 



FAX DOCUMENT 

TO: A CO. MR. a 

WE HAVE BEEN ORDERED 
REGARDING THE MAHER. 

TOM 



MR. TOM TRANSMITTED 
FILE ABC TO MR. a OF 
A CO.. LTD AT 10:00 AM 
ON JULY 5, 1996. 




COPY 
FILE ABC 



D 



7\ 



960 



1. 




171 



EP 0 853 301 A2 




1265 



172 



BNStXXJID: <EP. 



.0853301 A2J_> 



EP 0 853 301 A2 



FIG. 127 



CHANGE IN ENVIRONMENT, 
CHANGE IN STATUS OF TASK 
FROM OTHER EQUIPMENT 



USER'S OPERATION 



1 

t ^^ 


1 




42 

u 


TASK RECEIVE 




OPERATION 






UNIT 




UNIT 






1 












TASK ANALYSIS 


--45 


OPERATION 
DATA 






UNIT 








1 




TASK TABLE 








PLANNING 
DECISION UNIT 


--46 






1 










PROCESS 


--47 








EXECUTE UNIT 







INFORMATION PROCESSING 
APPARATUS 



1 



PAST SUCH AS INFORMATION 
TO ANOTHER EQUIPMENT 



TASK RECEIVE 
UNIT 

I 



OPERATION 
UNIT 



TASK TABLE 



TASK ANALYSIS 
UNIT 





PLANNING 
DECISION UNIT 








PROCESS 
EXECUTE UNIT 









INFORMATION PROCESSING 
APPARATUS 



7^ 



OPERATION 
DATA 



RESULTS OF EXECUTION SUCH AS 
FILING. PRINTING AND DISPLAY 



173 



08S3301A2_ I > 



EP 0 853 301 A2 



FIG. 128 



( START ;> 



INITIALIZE TASK TABLE 



If 



S51 




S53 



PREFERENTIAL TASK 



ANALYZE AIM OF TASK 



S56 



PLANNING • DECISION 



If 



S57 





EXECUTE PROCESS 
ACCORDING TO 
PUNNING 






ADD NEW TASK TO 
TASK TABLE 





S58 



S59 



BNSDOCID: <EP ^0853301 A2_l_> 



174 



EP 0 853 301 A2 



FIG. 



129A 

61 



START CONDITION 



IDLING FOR PREDETERMIND 
/I PERIOD OF TIME 



CONTENTS OF TASK 



MAKE( 

Object: IDLE PERIOD TASK ) 



FIG. 



62 



FIG. 129C 



63 



FIG. 129D 



64- 



65 



FIG. 



129E 

65 



FIG. 





START CONDITION 


CONTENTS OF TASK 


UNCONDITIONAL 


ANALYZE { 

Object: RECEPTION INFORMATION 
fPfint "contrad.doc" from Bj ) 


IDLING FOR PREDETERMIND 
PERIOD OF TIME 


MAKE( 

Object: IDLE PERIOD TASK ) 




START CONDITION 


CONTENTS OF TASK 


UNCONDITIONAL 


PRINT ( 

Object: "contract.doc" 
=PULL{ 
Actor: A 

Object: < contract.doc> 
lobject: B)) 


IDLING FOR PREDETERMIND 
PERIOD OF TIME 


MAKE( 

Object: IDLE PERIOD TASK ) 


♦ 


START CONDITION 


CONTENTS OF TASK 


UNCONDITIONAL 


PUU.( 
Actor: A 

Object: <contraadoc> 
lobject: B )) 


UNCONDITIONAL 


PRINT ( 

Object: < contract.doc > 
=PULL) 


IDLING FOR PREDETERMIND 
PERIOD OF TIME 


MAKE( 

Object: IDLE PERIOD TASK) 


♦ 


START CONDITION 


CONTENTS OF TASK 


UNCONDITIONAL 


PRINT { 

Object: < contract.doc> 
=PULL) 


IDLING FOR PREDETERMIND 
PERIOD OF TIME 


MAKE( 

Object: IDLE PERIOD TASK) 


♦ 



129F 

66 



START CONDITION 



IDLING FOR PREDETERMIND 
/I PERIOD OF TIME 



CONTENTS OF TASK 



MAKE( 

Object: IDLE PERIOD TASK ) 



175 



EP 0 853 301 A2 



FIG, 130 

KOSUGI 



NEW YORK 




SHIN KAWASAKI 

0 



SHIMOMARUKO 



MY FAX 



73- 



SET TRANSFER STATION 
AND CONDITIONS 
<TRANSFER $TATIONS > 
IPRINTER AT KC^DgT 




PLEASE ADVISE MY FAX TO TRANSFER 
ANY DOCUMENT SENT FROM MR. SUZUKI 
TO MR. TOYODA. 



OPERATION 
PANEL 



COPIER AT SHIMOMARUKO 

<GONDITION> 

fr 



72 



DESIGNATE DEVICE 
THAT YOU WANT 
TO OPERATE 




SHINKAWASAKI MY FAXl 
KOSUGI PRINTER 



UI FOR FAX 



UI FOR COPIER 



176 



BNSDOCID: <EP_ 



_0853301A2_L> 



EP 0 853 301 A2 



FIG. 131 

KOSUGl 





77 
PRINTER 



SHIN KAWASAKI 
HI I 




SHIMOMARUKO 



74 
COPIER 




SELECT 
NUMBER OF 
COPIES 

[1] COPY ^ 




COPY SEND 



UI FOR COPIER 




SELECT 
NUMBER OF 
COPIES 



[IjCOPY ^ 




OPERATION 
PANEL 34 



COPY SEND 



TRANSMIT DOCUMENT TO 
MR. a OF A CO., LTD 



OBTAIN TRANSMISSION 
FUNCTION 



SELECT 
DESTINATION 




A CO., LTD, MR. a 
B INC. . MS. b 
C CORP. , MISS, c 



UI FOR TRANSMISSION 



177 



0853301 A2_l_> 



EP 0 853 301 A2 



FIG. 132 



A111 CONFERENCE ROOM 





93 



DISPLAY PRESENT PLACE 
ON <MyDesk> 




91 

COPIER 




92 

2. 



7\ 



NOW IN CONFERENCE 

(SHIMOMARUKO A111 
CONFERENCE ROOM) 



//--/- / --[---\ 



<MyDesk> 



178 



EP 0 853 301 A2 



FIG. 133 




BNSDOCID: <EP 0853301A2_I_> 



179 



EP 0 853 301 A2 



FIG. 134 



115 



1- 



Contract-dx 



111 



1 



SCANNER 



FILING TO FILING SERVER 
TITLE, INDEX EXTRACTION 

113 




FILING SERVER 



BNSEXX^ID: <EP 0&53301A2_I_> 



180 



EP 0 853 301 A2 



FIG. 135 



121 



SCANNER 



ACTION: SEND 
WHAT: abc.doc 
FROM: FILING SERVER 
, TO: JOHN 

i> r 

123 




BNSDOCl D: < E P 085330 1 A2_t_> 



181 



EP 0 853 301 A2 



FIG. 136 

TASK = COMMUNICATION (A, B, INFORMATION) 



FIG. 136 A 



C START ^ 



CONTACT B 



S1312 



ANY REPLY 
, FROM B ? 

REPLY = 
JVILL CALLBACK 



^81313 
NO 



® 



YES 



NECESSARY TO 
J^TIFY USER ? 



^81319 
J^O 

1 



^81320 
JES 



FIG. 136A 



FIG. 136B 



NECESSARY TO 
.NOTIFY USER ? 



S1314 
YES 



NO 



81315 



NOTIFY 



'NECESSARY TO 
EXECUTE PRESENT 
vTASKAGAIN ? 




S1317 



ADD TASK TO 
TASK TABLE 



uNO 



DELETE TASK AND 
ADD TO FAILED LIST 



ISTp 



S1318 



NOTIFY 



If 



NO 



81321 



® 



ADD TASK TO 
TASK TABLE 
KEEP IN MIND OF 
UTER CALL BACK 



S1322 



182 



BNSDOCID: <EP ^0853301 A2_L> 



EP 0 853 301 A2 



FIG. 136B 



'REPLY = 




.CONTACT LATER ? 




,N0 


REPLY 


= 0K ? 




S1324 



S1331 ® 



^NO ^ 


DISPUY ERROR 






MESSAGE 





YES 



^81325 

/NECESSARY TO CONDUCtXyES. 



SI 326 



XNEGOTIATIONS ? 



NEGOTIATE 



,,N0 



TRANSMIT 
INFORMATION TO B 



SI 327 



ANY REPLY 
RECEPTION ? 



YES 



(C) /ANY NECESSARY' 
^ \ACTION ? 



^81328 
^NO 

JS1329 
NO 



YES 



ADD ACTION TO 
TASK TABLE 



SI 330 



C END ) 



183 



EP 0 853 301 A2 



FIG. 137 

TASK = PULL (A, B, INFORMATION) 



FIG. 137A 



FIG. 137B 



FIG. 137 A 



( START ^ 



CONTACT B 



ANY REPLY 
FROM B ? 



^81412 ^ 

S1413 
NO 



S1414 



uYES 



REPLY = 
JA/ILL CALLBACK 1, 



^81419 
.NO 



^Lw^NECESSARY TO^YES ' ^ 

^\NOTIFY USER ? 



S1415 



NOTIFY 



YES 



NO 



'NECESSARY TO 
EXECUTE PRESENT 
.TASK AGAIN ? 




SI 41 6 

YES 



S1417 



ADD TASK TO 
TASK TABLE 



NO 



S1420 



DELETE TASK AND l£ 
ADD TO FAILED LIST 



S1418 



/NECESSARY TO\YES^ ^rax^ \ 
V NOTIFY 1 LCiFR 1 ? NOTIFY f 




NO 




yS1422 


ADD TASK TO 
TASK TABLE 
KEEP IN MIND OF 
UTER CALL BACK 



S1421 



® 



184 



BNSDOCID: <EP 0853301A2_I_> 



EP 0 853 301 A2 



FIG. 137B 



'REPLY = 




(DONTACT LATER ? 




, NO 


REPLY 


= 0K ? 



SI 423 
YES 

SI 424 



S1431 ® 



>N0 , 


DISPLAY ERROR 






MESSAGE 





YES 



NECESSARY TO CONDUCT 
NEGOTIATIONS ? 




S1426 



NEGOTIATE 



NO 



TRANSMIT INFORMATION 
REQUISITION TO B 

, I 

/ANY INFORMATION' 
VRECEPTION ? 



SI 427 



YES 



(C) /ANY NECESSARY 
^ XACTION ? 



S1428 
NO 

jS1429 
NO 



uYES 



ADD ACTION TO 
TASK TABLE 



S1430 



C END ) 



185 



BNSDOCID: <EP ^0853301A2_I_> 



EP 0 853 301 A2 



FIG. 138 

TASK = PUSH (A, B, INFORMATION) 



FIG. 138A 



FIG. 138B 



FIG. 138 A 

C START ^ 



CONTACT B 



< 



S1512 



ANY REPLY 
FROM B ? 



^S1513 
.NO 



u YES 



REPLY = 
.WILL CA 



LBACK ?^ 



YES 



^S1519 
.NO 

I 



SI 520 



^S1514 

^NECESSARY TOX^ES 
.NOTIFY USER ? 



S1515 



NOTIFY 



NO 



'NECESSARY TO 
EXECUTE PRESENT 
gASK AGAIN ? 




S1516 
YES 



S1517 



ADD TASK TO 
TASK TABLE 



NO 



DELETE TASK AND 
ADD TO FAILED LIST 



S1518 



^NECESSARY TO\YES^ i 
V NOTIFY M.<5FR ? y> " NOTIFY \ 




NO 




^S1522 


ADD TASK TO 
TASK TABLE 
KEEP IN MIND OF 
LATER CALL BACK 



SI 521 



® 



186 



BNSDOCID: <EP ^0853301 A2_l_> 



EP 0 853 301 A2 



FIG. 138B 

1 



'REPLY = 




^CX)NTACT LATER I 




, NO 


REPLY 


= 0K ? 



S1523 
YES 

S1524 
NO 



® 



S1531 ® 



DISPLAY ERROR 
MESSAGE 



YES 



NECESSARY TO CONDUCT 
NEGOTIATIONS ? 



-S1525 
.YES. 



S1526 



NEGOTIATE 



I NO 



TRANSMIT NECESSARY 
INFORMATION TO B 



15^ 



SI 527 



ANY REPLY 
RECEPTION ? 



YES 



(C) /ANY NECESSARY 
^ XACTION ? 



S1528 
^10 

SI 529 
NO 



® 



YES 



ADD ACTION TO 
TASK TABLE 



15- 



S1530 



C END ) 



BNSDOCID: <EP 0e5330lA2j_> 



187 



EP 0 853 301 A2 



FIG. 139 

TASK = INDICATE {A, B, INSTRUCTION INFORMATION) 



FIG. 139A 



FIG. 139B 



F/G. 139 A 

( START ;> 



J .^-sieia 

CONTACT B r © 
S1613 



ANY REPLY 
FROM 8 ? 



NO 



S1614 



uYES 



REPLY = 
WILL CA 



LLBACK ?. 



YES 



NECESSARY TO 
NOTIFY USER ? 



^S1619 
NO 

1) 



^S1620 
.YES 



NECESSARY TO\YES. 
.NOTIFY USER ? 



S1615 



NOTIFY I 



NO 



'NECESSARY TO 
EXECUTE PRESENT 
sIASKAGAIN ? 




S1617 



ADD TASK TO 
TASK TABLE 



,rNO 



DELETE TASK AND 
ADD TO FAILED LIST 



S1618 



NOTIFY 



NO 



S1621 



ADD TASK TO 
TASK TABLE 
KEEP IN MIND OF 
LATER CALL BACK 



S1622 



188 



BNSDOCID: <EP ^08S3301A2J_> 



EP 0 853 301 A2 



FIG. 139B 

1 



REPLY = 




^CONTACT LATER I 




.NO 

[ — . ^ 


REPLY 


= 0K ? 



SI 623 
YES 

S1624 
NO 



SI 631 (b) 



DISPLAY ERROR 
MESSAGE 



YES 



-S1625 

NECESSARY TO CONDUCT^YES. 
NEGOTIATIONS ? 



S1626 



NEGOTIATE 



NO 



TRANSMIT INSTRUCTIONS 
INFORMATION TO B 



S1627 



ANY REPLY 
RECEPTION ? 



YES 



ANY NECESSARY 
ACTION ? 



S1628 
NO 

S1629 
NO 



YES 



ADD ACTION TO 
TASK TABLE 



81630 



( END ^ 



189 



EP 0 853 301 A2 



FIG. 140 

TASK = RESPOND (B, A, REQUEST) 



FIG. 140A 



FIG. 140B 



F\G. 140A 



( START ^ 



REQUEST TYPE 
= CO NTACT ? 

Tyes" 



JS1712 
NO 



S1719 



REQUEST TYPE WES 
INFORMATION REQUEST 



SI = PRESENT STATUS/ 
USER SET UP 



® 



S1713 



NO 



REQUEST TYPE 

= PUSH INFORMATION 



NO 



SI 721 

5^ 



REQUEST TYPE 
= INSTRUCTIONS ? 



NO 



SI 723 



S1720 



RESPOND 
/INFORMATION > 
REQUEST TYPE 

Vrequest ; 



SI 722 



RESPOND 
/INFORMATION \ 

STORAGE type 
VREQUEST / 



SI 724 



RESPOND 
/INSTRUCTIONS \ 
TYPE 

\INFORMATION / 



OPTIMAL REPLY 



SI 725 



® 



190 



BNSDOCID: <EP_ 



_0853301A2 I > 



EP 0 853 301 A2 



NO 



FIG. 140B 




1 



S^ = VERY BUSY/ 

STANDBY MODE ?. 



,rYES 



COMPLIANCE WITH 
REQUEST ? 



.S1714 



.S1715 
NO 



uYES 



S1716 



REPLY = WILL CALLBACK 



REPLY = OK 



81718 



81 71 7 



REPLY = CONTACT UTER 



( END ) 



BNSDOCID: <EP 0853301 A2_l_> 



191 



EP 0 853 301 A2 



FIG. 141 

RESPOND (TO B, A, INFORMATION REQUEST TYPE REQUEST) 



C START ) 



S1812 

NECESSARY TO CONDUCT\YES 
.NEGOTIATIONS ? 



81813 



NEGOTIATE 



1 


, NO 


FETCH INFORMATION 


1 





S1814 
S1815 



NO 



YES 



S1821 


< 


ADD TASK "TRANSFER- 
TO TASK TABLE 


81822 






REPLY = OK 



< TRANSFER IT AT ONCE ? 



YES 



NECESSARY TO 
FORMAT/CONVERT IT ? 



S1817 



81818 
NO 



81816 

4^ 

REPLY = NG 



YES 



81819 



CONVERT 
INFOR MATION 

zzu — 



TRANSMIT INFORMATION 



81 820 



C END ) 



BNSDOCID: <EP 0853301A2^L> 



192 



EP 0 853 301 A2 



FIG. 142 

RESPOND (TO B, A, INFORMATION STORAGE TYPE REQUEST) 



( START ) 



S1912 

NECESSARY TO CONDUCTXYES 
.NEGOTIATIONS ? 



S1913 



NEGOTIATE 





.NO 


ACQUIRE INFORMATION 







S1914 



ANY INFORMATION 
RECEPTION? 



NO 



YES 



S1916 



REPLY = NG 



NECESSARY TO 
FORMAT/CONVERT IT ? 



S1917 
NO 



1 S1918 



YES 



CONVERT 
INFORMATION 



J 



REPLY = OK 



S339 



( END / 



S340 



BNSDOCID: <EP ^0853301A2J_> 



193 



EP 0 853 301 A2 



FIG. 143 

RESPOND (TO B, A. INSTRUCTIONS TYPE REQUEST) 

( START ) 



S2012 

"necessary to CONDUCf\YES 
NEGOTIATIONS ? 



S2013 



negotiate 



Jno 



INTERPRETE INSTRUCTIONS^ 
INFORMATION 



S2014 



NO 



EXECUTABLE 
INSTRUCTIONS ? 

i"YES 



■S2015 
NO 



S2020 


\ 

r 


ADD TASK (EXECUTION) 
TO TASK TABLE 


S2021 






REPLY = OK 



1 


rYES 


EXECUTE instructions! 






REPLY = 

execution RESULT 



S2017 
S2018 



S2019 



( END ) 



S2016 

^n— , 

REPLY = NG 



194 



EP 0 853 301 A2 



FIG. 144 



DEVICE A 















r 1 



DEVICE B 




CONTACT B 



START 



OK 



REQUEST DATA <abc> 

[in <xyz format>] 
MTRANS 
{Actor: A 

Object: MTRANS { 
Actor: B 
Object: <abo 
lobject: A 

Support: <xyz format>} 
lobject: B...} 



FETCH DATA <abc> 



CONVERT FORMAT 
(IF NECESSARY) 



CONVERT FORMAT 
(IF NECESSARY) 



EXECUTE APPROPRIATE 
ACTION ACCORDING TO 
DATA 



END 



TRANSMIT DATA <abc> 
MTRANS { 
Actor: B 
Object: <abc> 
lobject: A 

Support: <xyz format>} 



BNSDOCID: <EP ^0853301 A2J_> 



195 



EP 0 853 301 A2 



DEVICE A 



CD 








1 \ 


i 1 



CONTACT B 



FIG. 145 



START 



DESKTOP B 




REQUEST DATA <abc> 
[in <xyz format>] 



OK 



OK 

FETCH DATA <abo 



OK 



CONVERT FORMAT 
(IF NECESSARY) 

EXECUTE APPROPRIATE 
ACTION ACCORDING TO 
DATA 



START 



END 



CONVERT FORMAT 
(IF NECESSARY) 

PRINT DATA <abc> 



196 



BNSDOCID: <EP_ 



_0853301A2J > 



EP 0 853 301 A2 



DEVICE A 




FIG. 146 



DEVICE B 




CONTACT B 



START 



OK 



OBTAIN FILE LIST 



TRANSMIT FILE LIST 



DETERMINE DATA <abc> 
TO BE PRINTED 



REQUEST DATA <abc> 
[in ocyz fomnat>] 



CONVERT FORMAT 
(IF NECESSARY) 



END 



FETCH DATA <abc> 

CONVERT FORMAT 
(IF NECESSARY) 

TRANSMIT DATA <abc> 



EXECUTE APPROPRIATE 
ACTION ACCORDING TO 
DATA 



197 



08S3301A2 1 > 



EP 0 853 301 A2 



FIG. 147 






' 








CD 


1 






PRINTER 



ACTION: 
WHAT: 



READ OUT 
NEW MAIL 



OK 



CANCEL 



FIG. 149 




198 



EP 0 853 301 A2 



FIG. 148 



DEVICE A 



/ — 


\ 


CD 








t \ 
( 1 



DESKTOP B 




CONTACT B 



START 



OK 



REQUEST DATA 

NEW MAIL IN <VOICE> 
MTRANS 
{Actor: A 

Object: MTRANS { 
Actor: B 

Object: New mail 
lobject: A 
Support: <Voice>} 
B...} 



lobject: 



EXECUTE APPROPRIATE 
ACTION ACCORDING TO 
DATA 



END 



FETCH NEW MAIL 



CONVERT 
VOICE FORMAT 



TRANSMIT DATA 



MTRANS { 
Actor: B 

Object: {Mail from John..} 
lobject: A 
Support: <Voice>} 



SPEAK OUT INFORMATION 



FROM MR. JOHN, 

MASSAGE "WHEN IS UI DESIGN CLEAR 
FOR ME ?" HAS BEEN MAILED. . 



199 



0853301 A2_L> 



EP 0 853 301 A2 





DOCUMENT READER 



ACTION: SEND & HOLD AT B 

WHAT: DOCUMENT SCANNED THIS MORNING 

WHOM: DEVICE B 



OK 



CANCEL 



200 



EP 0 853 301 A2 



FIG. 151 



DEVICE A 



DEVICE B 




^iPT \ 


CD 

1 




' 1 
r 1 



CONTACT B 



START 



OK 



REQUEST TRANSMISSION ^ 

OF DATA <abc> 

^ TRANSMIT 

[<xyz> format] 



CONVERT FORMAT 
(IF NECESSARY) 



TRANSMIT DATA <abc> 
[in <xyz format>l 
+ HOLD 



RECEIVE DATA 

CONVERT FORMAT 
(IF NECESSARY) 

EXECUTE APPROPRIATE 
ACTION (HOLD) 
ACCORDING TO DATA 



BNSDOCID: <EP ^0853301 A2J_> 



201 



EP 0 853 301 A2 



FIG. 152 



SCANNER 



FILING TO FILING SERVERS B AND C 
JITLE AND INDEX EXTRACTION 

>— ^ 




SCANNER C 

FILING SERVER 



202 



EP 0 853 301 A2 



FIG. 153 



DEVICE A DEVICE B 




OK 



TRANSMIT 
[<xyz> format] 



CONTACT B 5™L 



WISH TO TRANSFER 
INFORMATION <abc> 
TO BE FIELD FOR USER 1 



CONVERT FORMAT 
(IF NECESSARY) 

TRANSMIT DATA <abc> 

[in <xyz format>] 

& FILING TO FILING 
SERVER TITLE. 
INDEX EXTRACTION 



END 



RECEIVE DATA 

CONVERT FORMAT 
(IF NECESSARY) 



EXECUTE APPROPRIATE 
ACTION (FILING TO 
FILING SERVER TITLE, 
INDEX EXTRACTION) 
ACCORDING TO DATA 



203 



BNSWDCID: <EP 0853301A2_I_> 



EP 0 853 301 A2 



FIG. 154 




PRINTER 



ACTION: 

WHAT: 

WHOM: 



SEND MAIL 

GET DESIGN FROM BILL 
JOHN 



OK 



CANCEL 



204 



BNSDOCID: <EP 0853301 A2J_> 



EP 0 853 301 A2 



FIG. 155 



DEVICE B 



DEVICE A 



CONTACT B 



START 



NOW IN CONFERENCE 

<SHIMOMARUKO A111 
CONFERENCE ROOtb 




OK 



NOTIFY GIVING ^ 

INSTRUCTION 

TRANSMIT 

RECOGNIZABLE 

FORM AT 

CONVERT INSTRUCTION 
INTO RECOGNIZABLE FORMAT 
<xyz> type 
(IF NECESSARY) 



TRANSMIT INSTRUCTION 
DATA, fDISPLAY 
[NOW IN CONFERENCE 
<SHIMOMARUKO A111 
CONFERENCE ROOM>JJ 



END 



RECEIVE DATA 



EXECUTE APPROPRIATE 
ACTION TO DISPLAY 
(TNOW in CONFERENCE 
<SHIMOMARUKO A111 
CONFERENCE ROOM>J) 
ACCORDING TO DATA 



BNSDOCID: <EP ^0853301 A2J_> 



205 



EP 0 853 301 A2 



FIG. 156 



DEVICE A 




DEVICE B 



CONTACT B 



START 



OK 



NOTIFY GIVING 
INSTRUCTION 



CONVERT INSTRUCTION 
INTO RECOGNIZABLE FORMAT 
<xyz> type 
(IF NECESSARY) 



TRANSMIT 
RECOGNIZABLE 
FORM AT 
[<xyz> type] 



TRANSMIT INSTRUCTION _ 
DATA, rCONDITION SET UP 
riF DOCUMENT FROM MR.SUZUKI 
HAS JUST REACHED, THEN 
FORWARD IT TO MR. TOYODAJJ 



END 



RECEIVE DATA 



EXECUTE APPROPRIATE ACTION 
TO SET UP CONDITION THAT 
(ANY DOCUMENT SENT FROM 
MR. SUZUKI AND ADDRESSED 
TO USER SHOULD BE 
TRANSFERRED TO MR. TOYODA) 
ACCORDING TO DATA 



BNSDOCID: <EP p853301A2_l_> 



206 



EP 0 853 301 A2 



FIG. 157 



DEVICE A 



DEVICE B 



SCANNER 




CONTACT B 



START 



OK 



NOTIFY GIVING ^ 

INSTRUCTION 

TRANSMIT 

RECOGNIZABLE 

FORM AT 

CONVERT INSTRUCTION ^^^^"^ ^P®^ 

INTO RECOGNIZABLE FORMAT 

<xyz> type 

(IF NECESSARY) 



TRANSMIT INSTRUCTION — ^ RFrEIVF DATA 

DATA. [TRANSMIT "abc.doc" END KbUhiVb uaia 

TO MR. JOHNJ 

EXECUTE APPROPRIATE 
ACTION (TRANSMISSION 
OF "abc.doc" FROM FILING 
SERVER TO MR. JOHN) 
ACCORDING TO DATA 



0853301A2 l_> 



207 



EP 0 853 301 A2 



FIG. 158 



FILING SERVER 



PATENT 

PRODUCT 
SPECIFICATION 

OTHER CO. 
-h PRODUCT 
INFORMATION 



aaa.txt 



abcdocl 



ccc.txt 
xyz.doc 

A CO. 
B CO. 
- C CO. 
D CO. 



DISPLAY 
PRINT ^ 
TRANSMilt f 



DESTINATION 



INPUT DESTINATION 



JOHN 



352 




FILING SERVER 



JOHN'S DESKTOP 



208 



EP 0 853 301 A2 



FIG. 159 



PRINT WINDOW 



DISPLAY READABLE INFORMATION 
COMPUTER STORED MEMORY 

2^ 




INSTRUCT PRINTER X 
TO EXECUTE PRINTING 
BASED ON CONTENTS 
SET BY USER 





1^ \ 


I CD 




=1 


f 1 

( L 



209 



EP 0 853 301 A2 



FIG. 160 



REMOTE CONNECTION 

LOGIN: ichiro 
PASSWORD: 

LAST LOGIN: TUE FEB 4 16:03:18 FROM ABC 

YOU HAVE MAIL ! 
ichiro > 




BNSDOCID: <EP 0863301 A2J_> 



210 



EP 0 853 301 A2 



FIG. 161 



SCANNER Y 



COPY WINDOW (SCANNER Y) 



COPY 

DESTINATION 

NUMBER OF SETS 
TO BE COPIED 



PRINTER X 



SETS 



PRINT 



CANCEL 




PRINTER X 





'pn \ 


CD 




— 1 




r \ 



0853301A2_ I > 



211 



EP 0 853 301 A2 



FIG. 162 



PRINT JOB 



OUTPUT RESULTS 



391 



INPUT JOB 
JUDGMENT UNIT 



1 



396 



OUTPUT UNIT 





HOLD JOB 
MANAGEMENT UNIT 




OUTPUT WAIT JOB 
MANAGEMENT UNIT 



393 



HOLD 

JOB 

TABLE 




397 



OUTPUTTED JOB 
MANAGEMENT UNIT 



395 



OUTPUT 
WAITING 
JOB 
TABLE 




398 



OUTPUTTED 

JOB 

TABLE 



399 



TASK TABLE 



BNSDOCID: <EP ^0853301 A2_l_> 



212 



EP 0 853 301 A2 



FIG. 163 

PRINT (A, "abcdoc") OR PRINT (NOT A, "abc.doc") 



FIG.163A 



FIG. 163B 



F/G. 163 A 




213 



EP 0 853 301 A2 



FIG. 163B 




CONVERT 
DOCUMENT 



S4008 



INDICATE 
CONVERSION 



YES 




S4009 



S4010 



S4011 



PRINT BY 



•-^.^ESELF ?^ 
jpYES 




1 


r 


^54012 


PUSH & PRINT BY 
OTHER DEVICE 


PRINT DOCUMENT 







_S4013 




RESERVE PRINTED FILE 
AS PRINTED JOB EXECUTE 
OTHER PROCESS 



S4016 



214 

eNSOOCIO: <EP ^0853301 A2_l_> 



EP 0 853 301 A2 



FIG. 164 




SEND "abc.doc" 
TO PRINTER B ! 



PRINTER B 
412 



411 
SCANNER A 



/ 



PRINT HELD 
DOCUMENT 



RESERVE "abc.doc" AS HELD JOB 
(NOT PRINTED OUT IMMEDIATELY) 

SHOW HELD ' 
DOCUMENT ! 



"abc.doc" IS CHANGED 
TO PRINT WAITING JOB 




I 



"abcdoc" IS DISPUYED 
ON PANEL 



"abc.doc" IS PRINTED 




DELETE HELD 
DOCUMENT I 



"abc.doc" IS CHANGED 
TO PRINTED JOB 



"abc.doc" IS DELETED 
FROM HELD JOB 





DELETE PRINTED 
DOCUMENT I 



"abc.doc" IS DELETED 
FROM PRINTED JOB 



BNSDOCID: <EP ^0853301A2_I_> 



215 



EP 0 853 301 A2 



FIG. 165 




JOBS 4201 
OOBS HELD 

4202 
JOBS IN QUEUE 

JOBS IN HISTORY 
4203 



4204 



TITLE SENDER SOURCE PRIORITY STATUS 



JUPITER RAJNISH INNOVATION HIGH HELD 



4205 



4206 



4207 



PRINT 



HOLD 



EXTEND HOLD 



SEARCH 



SEND 



DELETE 



CHANGE PRIORITY 



SET INST. 



4208 



4209 



4210 



4211 



BNSDOCID: <EP 0853301A2_L> 



216 



(19) 



J) 



Europaisches Palentamt 
European Patent Office 
Office europeen des brevets 




(12) 



(11) EP 0 853 301 A3 

EUROPEAN PATENT APPLICATION 



(88) Date of publication A3: 

30.12.1998 Bulletin 1998/53 



(51) Intel 6: G08C 19/10, G06F 13/00, 
G05B 19/00, G08C 19/28 



(43) Date of publication A2: 

15.07.1998 Bulletin 1998/29 



(21) Application number: 97310582.8 



(22) Date of filing: 24.12.1997 



(84) 


Designated Contracting States: 


• Takayama, Masayuki 




AT BE CH DE DK ES Fl FR GB GR IE IT LI LU MC 


Ohta-ku, Tokyo (JP) 




NL PT SE 


♦ Suda, Aruna Rohra 




Designated Extension States: 


Ohta-ku, Tokyo (JP) 




AL LT LV MK RO SI 


• Wakai, Masanori 






Ohta-ku, Tokyo (JP) 


(30) 


Priority: 26.12.1996 JP 348308/96 


• Mikame, Shuichi 


28.02.1997 JP 44528/97 


Ohta-ku, Tokyo (JP) 




06.03.1997 JP 51718/97 


♦ Fuji!, KenichI 






Ohta-ku, Tokyo (JP) 


(71) 


Applicant: CANON KABUSHIKI KAISHA 


• TakahashI, Satomi 


Tokyo (JP) 


Ohta-ku, Tokyo ( JP) 


(72) 


Inventors: 


(74) Representative: 


• 


Jeyachandran, Suresh 


Beresford, Keith Denis Lewis et al 




Ohta-ku, Tokyo (JP) 


BERESFORD & Co. 


• 


Ibaraki, Shotchi 


2-5 Warwick Court 




Ohta-ku, Tokyo (JP) 


High Holborn 






London WC1R 5DJ (GB) 



(54) A remote control apparatus and a system therefor 



CO 
< 

o 

CO 
CO 

in 

00 

o 

UJ 



(57) A remote controller for exchanging information 
using infrared rays requests the direct transmission of 
a user interface (Ul) to a device to be controlled, such 
as a printer or a facsimile machine, or the indirect trans- 
mission through another device connected to the device 
to be controlled. When the Ul is returned upon the re- 
ceipt of the request, the menu of functions that the de- 
vice that is to be controlled can execute using the Ul is 
displayed on the screen of the remote controller, and a 
command to execute a function that is selected from the 
menu is transmitted to the device that is to be controller 



FIG. 1 




Printed by Jouve. 75001 PARIS (FR) 



EP 0 853 301 A3 



European Patent PARTIAL EUROPEAN SEARCH REPORT Appltcatton Number 

Office Which under Rule 45 Of the European Patent Conventlon^p 97 31 0582 

Shalt be considered, for the purposes of subsequent 
proceedings, as the European search report 



DOCUMENTS CONSIDERED TO BE RELEVANT 



Category 



Cttatton of document with indicabon. where appropriate, 
of retevant passages 



Relevant 
to claim 



CU^SIFtCATION OF THE 
APPLICATION (lnt.CI.6) 



EP 0 129 794 A (TELEFUNKEN FERNSEH UNO 
RUNDFUNK GMBH) 2 January 1985 



* abstract * 

* page 3, line 7 - 

* page 5, 1 ine 10 



line 20 * 

- line 25; figures 1-4 * 



PATENT ABSTRACTS OF JAPAN 
vol. 13, no. 348 (E-799), 4 August 1989 
i 3? 01 106635 A (MEC HOME ELECTI>ON 
LTD.). 24 April 1989 

* abstract * 

DE 39 27 838 A (DEUTSCHE THOMPSON-BRANDT 
GMBH) 28 February 1991 

* abstract * 

* column 1, line 51 - column 2, line 9; 
figure 1 * 

EP 0 564 201 A (CANON KABUSHIKI KAISHA) 
6 October 1993 

* abstract; figures 1,2 * 



1-7,11, 

19-25, 

32-38 



8-10, 

12-18, 

26-31 

1,2,4-6, 
8-10,20, 
22-25 
32-34' 



1-38 



39-174 



INCOMPLETE SEARCH 



Tho Soared Divtston consider that trie present appltcation. or one or mom oi ttscleims. does/do 
rot oompty with the EPC to such an extent that a meaningful search into the stale of the art cannot 
be earned out. or can onty be carried out paniaVy. for these c^ms. 

Clanm searched comptetety : 

1-38 

Claims searched incompletely : 

39-174 

Clams not searched : 

neafton for the limitation of the search: 

Considering EPC art. 84 and rule 45, CI. 39-174 lack 
ready comprehensibility. they are not concise in 
themselves. It is not possible to determine to which 
part of the disclosure they relate or if there is 
unity among them. 



G08C19/10 
G06F 13/00 
G05B 19/00 
S08C 19/28 



TECHNICAL FIELOS 
SEARCHED (lnt^.6) 



GOBC 

H04N 
H04B 



THE HAGUE 



Dale of completion of «te search 

28 October 1998 



EicBintnef 

C'Reilly, D 



CATEGORV OF CITED DOC JIVlENTS 

X : pan^cularty retevant H taken alone 

Y : particularty relevant it combirredw^h another 

document of the same category 
A : tecnrvologicat background 
O : non-written disdosure 
P : imemediate docunridm 



T : ttieory or principle underlying the invention 
E : earfier patent document, but pubfisfied on. or 

after the fitbtg date 
O : docum^t dted in the appltcation 
L : docajmert oted tor other reasons 

& : member of ttie same patent f^rnty, correspomfing 
document 



2 



BNSDOCID: <EP 085330 1A3_I_> 



EP 0 853 301 A3 



J) 



European Patent PARTIAL EUROPEAN SEARCH REPORT Application Number 

Office EP 97 31 0582 



DOCUMENTS CONSIDERED TO BE RELEVANT 



Category 



Citation of document with indication, where appropriate. 
of relevant passages 



GB 2 288 508 A (OFFSET STUDIOS LTD.) 
18 October 1995 

* abstract * 

* page 2, line 13 - page 4, line 21; 
figures 1,2 * 

US 5 528 374 A (MATIAS) 18 June 1996 

* abstract * 

* column 2, line 38 - line 53; figures 1-3 



39-174 



39-174 



Relevant 
to claim 



CLASSIFICATION OF THE 
APPLICATION tlnt.CI.6) 



TECHNICAL FIELDS 
SEARCHED (tnt.CI^) 



3 



BNSDOCID: <eP ^0853301 A3J_> 



THIS PAGE BUNK (USPTO) 



